noamaanMulla-03/transformers-issues huggingface dataset:
Increase the default max_length parameter when using TransfoXL & XLnet.
Author: mfuntowicz
Pull request: yes

This is needed because the prepended PADDING_TEXT constant is already bigger than the default max_length parameter on the generate method, thus leading to no token being generated.

Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>

Comments:
I don't think it is necessarly needed actually because the `max_length` is overwritten by the model's specific configs in this case. See XLNet config for example: https://s3.amazonaws.com/models.huggingface.co/bert/xlnet-base-cased-config.json. 

I would prefer to not set the `max_length` here, but let the user insert it if needed. After discussion with @thomwolf we decided to not hardcode parameter values in the pipelines.py file, but only set them via the `task_specific_params` in the configs.

I think the problem is that the task specific params are not overwritten correctly here because the task-name is not correct. 
It is important the the task-name `text-generation` is given to the pipeline, so that this line works as expected:
https://github.com/huggingface/transformers/blob/1ae132a07d7f294cf58cd50f7db8723d00e282de/src/transformers/pipelines.py#L400





I tried to explain that here as well:https://github.com/huggingface/transformers/pull/5086#issuecomment-645847015 and think we should actually not pass the task name to the `__init__` function of the pipelines, but change the variable `task` to a static class variable @julien-c @LysandreJik @mfuntowicz @thomwolf 

@patrickvonplaten Ok, thanks for the hints, I'll check the point you mentioned 👌 

Just a note that in the inference-api we *do* pass the correct "text-generation" task name, so there might be something else going on here.

Ok I'll check as well

I think I know what's going on - will do a PR that should fix it

This line in the inference api should not be called `causal-lm`, but `text-generation` IMO: https://github.com/huggingface/api-inference/blob/9cab899965d164f85c4961f0deafbc5034523e45/shared.py#L47 
This way the correct parameters would be loaded from the config.

But I would prefer to actually make the "task" name a static variable as is shown here:
https://github.com/huggingface/transformers/pull/5216

@julien-c @mfuntowicz 

Oh yes my bad @patrickvonplaten, this was actually on a branch: https://github.com/huggingface/api-inference/pull/3/files
Warning on `add_special_tokens`
Author: LysandreJik
Pull request: yes

Warning on `add_special_tokens` when passed to `encode`, `encode_plus` and `batch_encode_plus`

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2966?src=pr&el=h1) Report
> Merging [#2966](https://codecov.io/gh/huggingface/transformers/pull/2966?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/cc6775cdf5b20ad382613d3bdbf0dd8364d23219?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/2966/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/2966?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #2966      +/-   ##
==========================================
- Coverage   77.12%   77.11%   -0.01%     
==========================================
  Files          98       98              
  Lines       15977    15979       +2     
==========================================
  Hits        12322    12322              
- Misses       3655     3657       +2
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/2966?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/tokenization\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/2966/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdXRpbHMucHk=) | `90.48% <100%> (+0.02%)` | :arrow_up: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/2966/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `92.2% <0%> (-0.33%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/2966?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2966?src=pr&el=footer). Last update [cc6775c...7a09a54](https://codecov.io/gh/huggingface/transformers/pull/2966?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Better TensorFlow 2 examples
Author: LysandreJik
Pull request: yes
Labels: name: wontfix

This PR aims to improve the TensorFlow examples of the library. It aims to use many tf-specific paradigms, including but not limited to Keras API, AMP, XLA, `tensorflow_datasets`, `tf.train.Features` and `tf.train.Examples`, model saving using Keras callbacks.

Two examples are specifically targeted:

- [x]  `run_tf_glue.py`
- [ ]  `run_tf_squad.py`

Aims to have a similar architecture to the PyTorch examples: using a parser with similar arguments so as to be used from a terminal.

Feedback is most welcome.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/1861?src=pr&el=h1) Report
> Merging [#1861](https://codecov.io/gh/huggingface/transformers/pull/1861?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/a5a06a851e1da79138e53978aa079a093f243dde?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/1861/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/1861?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #1861   +/-   ##
=======================================
  Coverage   81.43%   81.43%           
=======================================
  Files         122      122           
  Lines       18338    18338           
=======================================
  Hits        14933    14933           
  Misses       3405     3405
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/1861?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/1861?src=pr&el=footer). Last update [a5a06a8...0f730de](https://codecov.io/gh/huggingface/transformers/pull/1861?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
TensorFlow XLM doesn't accept NumPy arrays for the attention mask
Author: LysandreJik
Pull request: yes

Convert NumPy attention mask to a TensorFlow tensor so that the mask creation doesn't crash

closes #2729 

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2736?src=pr&el=h1) Report
> Merging [#2736](https://codecov.io/gh/huggingface/transformers/pull/2736?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/9c67196b83a824df577742d32d38e9121d8a9285?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/2736/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/2736?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #2736      +/-   ##
==========================================
+ Coverage   74.09%   74.09%   +<.01%     
==========================================
  Files          93       93              
  Lines       15249    15251       +2     
==========================================
+ Hits        11298    11300       +2     
  Misses       3951     3951
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/2736?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_xlm.py](https://codecov.io/gh/huggingface/transformers/pull/2736/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl94bG0ucHk=) | `90.47% <100%> (+0.06%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/2736?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2736?src=pr&el=footer). Last update [9c67196...3c9a47e](https://codecov.io/gh/huggingface/transformers/pull/2736?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Hi, any update on this PR?

After discussing it with @thomwolf, it seems I was mistaken when believing that our TensorFlow models should accept numpy inputs. They should be converted to TensorFlow inputs. We should update the documentation to reflect this. Closing this PR as unrelated to the doc changes.
Upgrade run_generation
Author: LysandreJik
Pull request: yes

The script `run_generation` has a few issues that I aim to fix in this PR:

- [x] The XLNet and XLM generations are broken (crash)
- [x] An end of sequence token is added to all sequences, even when models don't have that token, and results in weird end of sequences.
- [x] No way to generate multiple sequences at a time as it was possible before
- [x] The `length` parameter doesn't take into account the prompt length.
- [x]  The prompt is concatenated to the generated sequence, which results in concatenating the initial text for XLNet.
- [x] Actually implement languages for XLM

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2749?src=pr&el=h1) Report
> Merging [#2749](https://codecov.io/gh/huggingface/transformers/pull/2749?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/ada24def22199459d8c1decc311dfe8dae7a7d8c?src=pr&el=desc) will **decrease** coverage by `0.02%`.
> The diff coverage is `25%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/2749/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/2749?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #2749      +/-   ##
==========================================
- Coverage    75.1%   75.07%   -0.03%     
==========================================
  Files          93       93              
  Lines       15249    15255       +6     
==========================================
  Hits        11452    11452              
- Misses       3797     3803       +6
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/2749?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/2749/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `60.7% <0%> (-0.63%)` | :arrow_down: |
| [src/transformers/configuration\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/2749/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX3V0aWxzLnB5) | `96.46% <100%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/2749?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2749?src=pr&el=footer). Last update [ada24de...f2bcc91](https://codecov.io/gh/huggingface/transformers/pull/2749?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Changes have been added in PR #2885.
Tf qa pipelines test
Author: mfuntowicz
Pull request: yes


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2994?src=pr&el=h1) Report
> Merging [#2994](https://codecov.io/gh/huggingface/transformers/pull/2994?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/8194df8e0cff8e5866ec2bcbda34e3892f10eb39?src=pr&el=desc) will **increase** coverage by `0.06%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/2994/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/2994?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #2994      +/-   ##
==========================================
+ Coverage   77.16%   77.22%   +0.06%     
==========================================
  Files          98       98              
  Lines       15999    15999              
==========================================
+ Hits        12345    12355      +10     
+ Misses       3654     3644      -10
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/2994?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/2994/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `92.37% <0%> (+0.16%)` | :arrow_up: |
| [src/transformers/pipelines.py](https://codecov.io/gh/huggingface/transformers/pull/2994/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9waXBlbGluZXMucHk=) | `71.64% <0%> (+0.75%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_auto.py](https://codecov.io/gh/huggingface/transformers/pull/2994/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9hdXRvLnB5) | `74.5% <0%> (+5.88%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/2994?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2994?src=pr&el=footer). Last update [8194df8...0e7d609](https://codecov.io/gh/huggingface/transformers/pull/2994?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


@julien-c the TF_QA pipeline test currently fails with a segmentation error, and no way to debug it. We think it might be hardware related, hence the reduction in concurrency.

I think you inadvertently re-added `run_all_tests_torch_and_tf` (which should not be here anymore)

Maybe just mark the flaky test as `@slow`, and we'll see if it works more reliably on our own CI?

oops, indeed, my bad. Alright, that works as well.
Model templates
Author: LysandreJik
Pull request: yes

This PR adds a `cookiecutter`-based utility to generate configuration/modeling/tokenization files, the test suites and the statements across the library necessary for adding a new model.

This PR's goal is to make adding a new model way simpler, by having a simple CLI request information, generate files, that will then need to be edited to implement the changes relative to BERT. The test suites are implemented and run.

Left to do:
- [x] TensorFlow files
- [x] Tokenizer files
- [x] Remove the pooler from the base model
- [x] Ensure the documentation has the right format + .rst file
- [x] Clean-up/refactor the `add_new_model.py` file
- [x] Clarify and add comments to the "dark arts" parts of this PR, such as the `to_replace` file.
- [x] Add encoder-decoder models:
  - [x] Modeling PT file
  - [x] Configuration file
  - [x] Testing the modeling PT file
  - [x] Modeling TF file
  - [x] Testing the modeling TF file
  - [x] Update the RST with the appropriate files
  - [x] Add to all auto + init files
- [x] Run the CI on generated files
- [x] Update the LysBERT proposal to something better
- [x] Do a checklist of things left to do after running the script

Possible improvements:
- [ ] Ask the user whether they want to support `token_type_ids`

## For reviewers

If you review this PR, the simplest would be to review the `add_new_model.py` file, and to generate model files using the utility:
```
transformers-cli add_new_model
```
And review the generated files.
Reviewing the current files with `{{cookiecutter.lowercase_modelname}}` doesn't seem reviewer-friendly to me.

Comments:
BART CausalLM example
Author: LysandreJik
Pull request: yes

\

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Improving generation
Author: thomwolf
Pull request: yes

Fix #2554 

TODO:
- add tests on generation

TODO potential:
- this PR could be used to fix #2415 and fix #2482 as well
- add TF 2.0 support for generation

Comments:
It would be great if this PR could handle the padding index for the models that do not have one. For example, GPT-2 doesn't have a padding index and therefore can't use the `generate` method, nor can it use the `batch_encode_plus` method.

PR #2885 added the proposed changes.
standardize CTRL BPE files - upload models to S3
Author: thomwolf
Pull request: yes

This PR:
- update CTRL BPE files (`vocab.json` and `merges.txt`) to use a single format for sub-word splitting (selected to use `</w>` at the end of words)
- upload CTRL updated vocabulary files and pytorch model (not updated) to AWS.

cc @mfuntowicz 

Comments:
Also cc @keskarnitish!
Partially fix space only input without special tokens added int the output 
Author: mfuntowicz
Pull request: yes
Labels: name: Core: Tokenization

Original issue #3091.

It fixes the issue for all non BPE-based tokenizers. For BPE ones, the output is different from Python and Rust: 

GPT2: 
- Python : `[]`
- Rust: `['Ġ']`

Roberta:
- Python: `[]`
- Rust: `['
', 'Ġ', '
']`

Rust seems the right one here. I should have a look at Roberta why it include the special_tokens even if not asked to do so.

cc @n1t0 cc @LysandreJik fyi.

Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>

Comments:
[Don't merge] Run make style on templates
Author: JetRunner
Pull request: yes

When run `make style` locally, these files got modified. 

I noticed that `black` and `isort` seem to have conflicts on these files. Is there a solution?

```
> make style
black --line-length 119 --target-version py35 examples templates tests src utils
reformatted /Users/canwenxu/transformers/src/transformers/__init__.py
reformatted /Users/canwenxu/transformers/templates/adding_a_new_example_script/run_xxx.py
reformatted /Users/canwenxu/transformers/templates/adding_a_new_example_script/utils_xxx.py
All done! ✨ 🍰 ✨
3 files reformatted, 339 files left unchanged.
isort --recursive examples templates tests src utils
Fixing /Users/canwenxu/transformers/templates/adding_a_new_example_script/run_xxx.py
Fixing /Users/canwenxu/transformers/templates/adding_a_new_example_script/utils_xxx.py
Fixing /Users/canwenxu/transformers/src/transformers/__init__.py
```

Comments:
It seems to be an isort version problem
[ModelOutput] Proposal to fix compatibility issue with torch.DataParallel
Author: thomwolf
Pull request: yes

This is a proposal to fix #5693 by making `ModelOutput` inherit from a dictionary and behave like a dictionary on iteration (i.e. iterate over the keys rather than the values).

This could break backward compatibility when users iterates over the output tuple rather than indexing it.

On the other hand, we regain backward compatibility with `torch.DataParallel` and from a more general design point of view the `ModelOutput` class should probably be closer to a dictionary than a tuple in the future.

Comments:
Update: this is actually not working because it breaks the possibility to unpack the output of the model's forward pass which is obviously a common pattern (cf failure in the tests) :-/

This is superseded by #6138 now.
ONNX conversion script.
Author: mfuntowicz
Pull request: yes

This PR adds conversion script to export our models to ONNX IR. 

Plan is to support both PyTorch and TensorFlow: 

- [x] PyTorch
- [ ] TensorFlow

TensorFlow currently blocked because of an issue in the conversion script provided by ONNX (seems fixed on master : https://github.com/onnx/tensorflow-onnx/issues/876)

Comments:
Optimize PyTorch's GPT2 computation graph 
Author: mfuntowicz
Pull request: yes
Labels: name: wontfix

Brings various optimizations to make the computation graph more efficient.

Comments:
GPT2 Attention has `merge_heads` method which takes an input as `(batch, heads, seq, repr)` and outputs the flattened _(merged)_ representation of it `(batch, seq, heads * repr)`. 

```python
    def merge_heads(self, x):
        x = x.transpose(2, 1).contiguous()
        new_x_shape = x.size()[:-2] + (x.size(-2) * x.size(-1),)
        return x.view(*new_x_shape)  # in Tensorflow implem: fct merge_states
```

This can be simplified with `flatten()` to avoid constructing a tuple and accessing shape information.

```python
    def merge_heads(self, x):
        x = x.transpose(2, 1).contiguous()
        return x.flatten(-2)  # in Tensorflow implem: fct merge_states
```

GPT2 Attention has a `Conv1D` projection which is nothing else than a linear projection of the input. 

The current implementation use BLAS acceleration function `addmm` (Matrix Multiplication + Add) to handle the affine projection `w * x.T + b`. 

This is currently implemented with the non batched BLAS call and requires flattening the sequence over the batch axis and unflattening after the call: 

```python
def forward(self, x):
     size_out = x.size()[:-1] + (self.nf,)
     torch.addmm(self.bias, x, self.weight)
     return x.view(*size_out)
```

We can optimize this call to remove all the reshape operations by leveraging the batched BLAS call (`baddbmm` : Batched Matrix Multiply + Batched Add) which takes 3D inputs as (`b, m, n)` and thus avoid the reshapings

```python
def forward(self, x):
    return torch.baddbmm(self.bias[None, None], x, self.weight[None])
```


Use tuples instead of list for returning objects from layers (refer to [this SO post](https://stackoverflow.com/a/22140115))

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Attempt to do some optimizations for BERT models
Author: mfuntowicz
Pull request: yes
Labels: name: wontfix

- Use functional as much as possible instead of creating a class instance everytime
- Precompute and store the attention scaling factor to avoid `1/sqrt(...)` every forward
- Refactor Self-Attention to group QKV weights and and increase hardware density

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.