noamaanMulla-03/transformers-issues huggingface dataset:
PretrainedTokenizer cleanup: Typehints, decode_batch
Author: sshleifer
Pull request: yes

This is very very minor cleanup

- adds `decode_batch` which calls `.decode` on every entry in a list.
- A few cosmetic changes to tokenization_utils.py (type hints using defaultdict)
- adds type hints in files I touched.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3837?src=pr&el=h1) Report
> Merging [#3837](https://codecov.io/gh/huggingface/transformers/pull/3837?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/f399c00610506325bc1690f0e68c6885e73395ec&el=desc) will **decrease** coverage by `0.00%`.
> The diff coverage is `84.21%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3837/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3837?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3837      +/-   ##
==========================================
- Coverage   78.48%   78.48%   -0.01%     
==========================================
  Files         106      106              
  Lines       17930    17934       +4     
==========================================
+ Hits        14072    14075       +3     
- Misses       3858     3859       +1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3837?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/tokenization\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3837/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdXRpbHMucHk=) | `87.50% <82.35%> (-0.07%)` | :arrow_down: |
| [src/transformers/tokenization\_xlm.py](https://codecov.io/gh/huggingface/transformers/pull/3837/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25feGxtLnB5) | `83.46% <100.00%> (ø)` | |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3837/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `92.76% <0.00%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3837?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3837?src=pr&el=footer). Last update [f399c00...41502fa](https://codecov.io/gh/huggingface/transformers/pull/3837?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
How to fine-tune DialoGPT with your own data?
Author: avinregmi
Pull request: no

# 🚀 How to fine-tune DialoGPT with your own data?

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

## Motivation

I really liked the new DialoGPT that allows to make chatbots, but how do i fine-tune this for own dataset? 



Comments:
DialoGPT is fine-tuned in the same way GPT2 in fine-tuned. Please take a look at the paper: 
https://arxiv.org/pdf/1911.00536.pdf . 

I will also add some training tips to the doc string.
run_generation.py with empty input
Author: r0levrai
Pull request: no
Labels: name: Ex: Generation

Hi,

I would like to generate text following some context but also from scratch as seen in *Write with Transformer*. Using both
* `python run_generation_callable.py --model_type=gpt2 --model_name_or_path=gpt2` and feeding an empty prompt to the `Model prompt >>> `
* or changing the line
`prompt_text = args.prompt if args.prompt else input("Model prompt >>> ")`
in the source to
`prompt_text = args.prompt`
and using
`python run_generation_callable.py --model_type=gpt2 --model_name_or_path=gpt2 --prompt ""`

result in a `RuntimeError: cannot reshape tensor of 0 elements into shape [-1, 0] because the unspecified dimension size -1 can be any value and is ambiguous` (tell me if you need the complete stack trace or more detail for the reproduction).

Thanks by advance for any tips, workarounds or directions on achieving this!

Comments:
Hi @r0levrai, 

sorry for responding so late. Thanks for spotting the error. Once the linked PR is merged generation with an empty prompt should be fine! 

Also note that there is a generation pipeline now which you could use as follows (once the PR is merged):

```python
from transformers import pipeline
generator = pipeline("text-generation")
generator("") # empty prompt
```

This is good news, thanks!
Fix #3954 - GPT2 is not traceable
Author: jazzcook15
Pull request: yes

This PR replaces the `math.sqrt(...)` computation with `(...)**0.5` which allows the model to be correctly traced by `torch.jit.trace`.

Comments:
Need to remove the `math` import to pass the Flake test.

Just deleted the math import to make the code quality check happy - Thanks a lot for the PR @jazzcook15 and for linking the issue @minimaxir
License information by model
Author: alexcombessie
Pull request: no

Hi,

First of all, thanks for the good work. Very useful.

Would it be possible to add the license information for each model listed on https://huggingface.co/transformers/pretrained_models.html?
The reason is that for production, I need to know which models can be bundled in my app. Some licenses do not allow bundling...

I may have missed it, but I could not find licensing information in the doc or code.

If that information is not centralized, I am happy to do the research myself (and share results!). I would be interested in hints if you have some.

Cheers,

Alex

Comments:
Good question. As far as I can tell (I don't think there's a definitive a.k.a legally tested answer to that question) a model is not considered as a derivative work from the dataset(s) it was trained on, so the person who trained the model can choose whatever licensing option they want.

For instance, the original [BERT weights](https://github.com/google-research/bert) mention: 
> We will not be able to release the pre-processed datasets used in the paper.
> [...]
> These models are all released under the same license as the source code (Apache 2.0).

Please share your findings if you conduct more extensive research.

Thanks. That's what I assumed as well. Based on the linked you shared, it seems that all BERT models are under the same license. If that's the case for other model architectures, the investigation should be simple. I will look at the ~15 architectures supported and share my findings this week.

I have a separate question on language support by model, but I will submit it as a separate issue.

Have a great day,

Alex

Hi @julien-c,

I really liked your suggestion on #3397 to add it to model cards.

Could I add license information in the same way, using tags on the model card?

Cheers,

Alex

Yes @alexcombessie feel free to do some research and open a PR. You can add a `license: x` tag to the metadata, where `x` is an identifier found in https://help.github.com/en/github/creating-cloning-and-archiving-repositories/licensing-a-repository

A few additional data points:
- Camembert: MIT (source: https://camembert-model.fr/), trained on Oscar (https://traces1.inria.fr/oscar/) whose license is `cc0`
- same for all models from Fairseq: https://github.com/pytorch/fairseq#license

FYI @MobiusLooper

The Distil* models trained at Hugging Face are released under Apache 2.0.
Allow a more backward compatible behavior of max_len_single_sentence and max_len_sentences_pair
Author: thomwolf
Pull request: yes

Since #3706, `max_len_single_sentence` and `max_len_sentences_pair` are now automatically setup in the base class.

This PR allows the user to try to setup `max_len_single_sentence` and `max_len_sentences_pair`. If the value if the same as the pre-computed, we display a deprecation warning (avoid breaking old code in this case). If the value is different we raise an explicit error message.

cc @HendrikStrobelt

Comments:
[Fix common tests on GPU] send model, ids to torch_device
Author: sshleifer
Pull request: yes


Comments:
~@patrickvonplaten 
 any idea why tf tests dont run? 
https://app.circleci.com/pipelines/github/huggingface/transformers/5734/workflows/7986a344-3a5f-4f4f-a5e7-f8fd82ef7a2d/jobs/33381/steps
seems like test_modeling_common.py only used by torch.~

resolved: had a `PretrainedConfig` type hint 😲 

Great, thanks for fixing this :-)
Remove jitted method so that our models are pickable.
Author: LysandreJik
Pull request: yes

closes https://github.com/huggingface/transformers/issues/4038

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4050?src=pr&el=h1) Report
> Merging [#4050](https://codecov.io/gh/huggingface/transformers/pull/4050?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/9a0a8c1c6f4f2f0c80ff07d36713a3ada785eec5&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4050/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4050?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4050   +/-   ##
=======================================
  Coverage   78.87%   78.88%           
=======================================
  Files         111      111           
  Lines       18536    18533    -3     
=======================================
- Hits        14621    14620    -1     
+ Misses       3915     3913    -2     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4050?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/activations.py](https://codecov.io/gh/huggingface/transformers/pull/4050/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9hY3RpdmF0aW9ucy5weQ==) | `88.88% <ø> (+7.93%)` | :arrow_up: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4050/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.02% <0.00%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4050?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4050?src=pr&el=footer). Last update [9a0a8c1...16e9e66](https://codecov.io/gh/huggingface/transformers/pull/4050?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Adapter-BERT is missing in transformers library?
Author: ghost
Pull request: no
Labels: name: wontfix

Adapter BERT obtain comparable results to BERT on several NLP tasks while achieving parameter efficiency. ( https://github.com/google-research/adapter-bert ) @thomwolf 
I think, it will be useful if adapter-bert is also included in the library.



Comments:
Up, I think this is an awesome idea

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Evaluation and Inference added to run_glue.py
Author: CMobley7
Pull request: no
Labels: name: wontfix

name: Ex: Sequence Classification

# 🚀 Feature request

It would be useful to have the following arguments added to `run_glue.py`, as well as probably the other task example scripts: `--eval_only` and `--inference_only`.

## Motivation

This will allow users to either provide a `.tsv` or `.csv` with either sentences and labels or just sentences and then perform evaluation or inference on the model without having to train. This may already exist, but I was unable to find it. Consequently, I'm using https://github.com/kaushaltrivedi/fast-bert for these tasks. It would be helpful to not have to stray from the main huggingface transformers library.


Comments:
Hi! You can already provide `do_eval` to `run_glue` to do the evaluation. If you don't specify `do_train`, it will only do the evaluation, and no training.

The inference would be a nice addition.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
How to get the matrix that is used to combine output from multiple number of attention heads?
Author: h56cho
Pull request: no
Labels: name: Help wanted

name: wontfix

Hello,

if I am understanding transformers correctly, right before the feedforward layer, output of individual attention head are concatenated and multiplied by a matrix **H**, so that the outputs from the multiple number of heads will be combined into one output which will then be an input to the subsequent feedforward block within the same layer.

Is there any way that I can retrieve the matrix **H** from the Hugging Face GPT2 model?

Thank you,

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
How to train with variable number of candidates for multiple choice selection?
Author: g-karthik
Pull request: no
Labels: name: Help wanted

name: wontfix

name: Usage

# ❓ Questions

## Details
I am trying to train `GPT2DoubleHeadsModel` for the tasks of generation and multiple-choice selection. My dataset has examples with variable number of candidates - some examples have 10 candidates, some have 15, etc.

I wanted to be able to create a single `TensorDataset` object for my dataset and train the model using a `DataLoader` wrapped around this dataset. But clearly, since the number of candidates varies across examples, I am unable to do so.

What is an appropriate way (or best practice) to train `GPT2DoubleHeadsModel` with such a dataset?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
OpenAIGPTDoubleHeadsModel throws CUDA OOM with large number of candidates
Author: g-karthik
Pull request: no
Labels: name: wontfix

name: Core: Modeling

# 🐛 Bug

I am trying to train the `OpenAIGPTDoubleHeadsModel`. I find that large number of candidates can cause CUDA OOM errors.

Case 1 (single training example with 67 candidates): CUDA OOM
```
input_ids.shape: torch.Size([1, 67, 275])
mc_token_ids.shape: torch.Size([1, 67])
lm_labels.shape: torch.Size([1, 67, 275])
mc_labels.shape: torch.Size([1])
token_type_ids.shape: torch.Size([1, 67, 275])
```

Case 2 (single training example with 3 candidates): works fine!
```
input_ids.shape: torch.Size([1, 3, 275])
mc_token_ids.shape: torch.Size([1, 3])
lm_labels.shape: torch.Size([1, 3, 275])
mc_labels.shape: torch.Size([1])
token_type_ids.shape: torch.Size([1, 3, 275])
```

## Information

Model I am using: `OpenAIGPTDoubleHeadsModel`

Language I am using the model on: English

The problem arises when using my own modified scripts based on the `transfer-learning-conv-ai` repo by Hugging Face.

## To reproduce
Simply try training `OpenAIGPTDoubleHeadsModel` with larger number of candidates (such as 67).

## Expected behavior

3 or 67 candidates shouldn't matter, both cases 1 and 2 should work fine without CUDA OOM.

## Environment info
     
- `transformers` version: 2.3.0
- Platform: Amazon Linux (Deep Learning AMI)
- Python version: 3.6
- PyTorch version (GPU?): the one shipped with the pytorch_p36 conda env in Amazon DL AMI
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Yes and No


Comments:
Isn't that to be expected? The large model just doesn't fit on your GPU's memory.

@BramVanroy is that expected behavior with just 1 training example though?

I initially suspected that this is a training-specific behavior (due to the need to store gradients, etc.), so I decided to fix the number of candidates to something small during training.

I then attempted to do inference with this trained model, but I used all candidates during inference. I took 1 example to do inference on, and I observed memory issues with that too.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Fix for case of multi-gpu
Author: orena1
Pull request: yes
Labels: name: wontfix

When loading the optimizer and the scheduler in a multi gpu, the loading will put the optimizer and scheduler in cuda:0, it might not have enough mem to temporary store them (until the to.device below).

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Language generation not possible with roberta?
Author: ysig
Pull request: no

Hi,

I have a small question.
I finetuned a roberta model and then I was about to do generation and I understood this was not possible on my model, but only in a smaller collection of model, where most of them are quite huge.
Why is that? Is there another way to generate language with roberta?

Thanks in a advance!

PS: the scripts I used were: `run_language_modeling.py` and `run_generation.py`

Comments:
`Roberta` has not really been trained on generating text. Small models that work well for generation are `distilgpt2` and `gpt2` for example.
Hoist bert model tester for patrick
Author: sshleifer
Pull request: yes


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4027?src=pr&el=h1) Report
> Merging [#4027](https://codecov.io/gh/huggingface/transformers/pull/4027?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/4e817ff41885063e08bb3bcd63e5adfd835b9911&el=desc) will **decrease** coverage by `0.07%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4027/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4027?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4027      +/-   ##
==========================================
- Coverage   78.44%   78.37%   -0.08%     
==========================================
  Files         111      111              
  Lines       18518    18518              
==========================================
- Hits        14527    14513      -14     
- Misses       3991     4005      +14     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4027?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4027/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `90.46% <0.00%> (-2.31%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4027?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4027?src=pr&el=footer). Last update [4e817ff...7189fff](https://codecov.io/gh/huggingface/transformers/pull/4027?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Fixed Style Inconsistency
Author: jtaylor351
Pull request: yes

This fixes a style inconsistency in BertForSequenceClassification's constructor where 'config' is referenced using both the parameter 'config' and 'self.config' on the same line.

This line is the only instance in modeling_bert.py where the config parameter is referenced using self.config in a constructor.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3976?src=pr&el=h1) Report
> Merging [#3976](https://codecov.io/gh/huggingface/transformers/pull/3976?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/4e817ff41885063e08bb3bcd63e5adfd835b9911&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3976/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3976?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3976      +/-   ##
==========================================
- Coverage   78.44%   78.43%   -0.02%     
==========================================
  Files         111      111              
  Lines       18518    18518              
==========================================
- Hits        14527    14525       -2     
- Misses       3991     3993       +2     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3976?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_bert.py](https://codecov.io/gh/huggingface/transformers/pull/3976/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19iZXJ0LnB5) | `88.40% <100.00%> (ø)` | |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3976/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `68.49% <0.00%> (-0.37%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3976/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `92.59% <0.00%> (-0.17%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3976?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3976?src=pr&el=footer). Last update [4e817ff...6e5ef43](https://codecov.io/gh/huggingface/transformers/pull/3976?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Thanks @jtaylor351 !
Dropout training
Author: weiarqq
Pull request: no

# ❓ Questions & Help

outputs = self.bert(inputs, **kwargs)

pooled_output = outputs[1]

pooled_output = self.dropout(pooled_output, training=kwargs.get("training", False))
logits = self.classifier(pooled_output)
outputs = (logits,) + outputs[2:]  # add hidden states and attention if they are here

I don't see any place where training of dropout are being used in examples, I was puzzled.So can someone tell me about this?



Comments:
Hi @weiarqq, 

Where does the code come from? What is the application? What do you mean by "training of dropout"? the `training=kwargs.get("training", False)` is a flag for pytorch's dropout to switch it on and off depending on whether you are in training module "training=True" or not.