noamaanMulla-03/transformers-issues huggingface dataset:
update min tf requirements
Author: stas00
Pull request: yes

All of the test suite is failing w/o this update - need to re-run `pip install -e .[dev]`

note that the failing tests don't show `"You need to run the TensorFlow trainer with at least the version 2.2.0, your version is {` anywhere so perhaps need some extra tweaks for the test fixtures, but then since non-tf tests fail too, the problem is in the core.

Not sure if perhaps this code needs to be replaced with an assert - then the error message will always be there regardless where it's used.
```
if parse(tf.__version__).release < (2, 2, 0):
    logger.info(
        "You need to run the TensorFlow trainer with at least the version 2.2.0, your version is {}".format(
            tf.__version__
        )
    )
    sys.exit(1)
```

Currently, if I run **any** test, including pytorch-only tests, with tf < 2.2 I get:

```
____________________________________________________________ ERROR collecting tests/test_benchmark.py ____________________________________________________________
tests/test_benchmark.py:6: in <module>
    from transformers import AutoConfig, is_torch_available
src/transformers/__init__.py:659: in <module>
    from .trainer_tf import TFTrainer
src/transformers/trainer_tf.py:34: in <module>
    sys.exit(1)
E   SystemExit: 1
======================================================================== warnings summary ========================================================================
```

To reproduce:

```
pip install tensorflow==2.0.1
pytest -ra tests/test_benchmark.py 
```

@jplu 



Comments:
Hello !

Which tests do not pass precisely? They are all green for me.

For the log that is not displayed this is because no logger is set in `benchmark_test.py`. I will review that for later, thanks!

> Which tests do not pass precisely? They are all green for me.

Most (all?) tests. Try:
```
pip install tensorflow==2.0.1
pytest -ra tests/test_benchmark.py 
```

But unrelated to tests if the runtime requires `foo>=x.y.z`, then the requirements/setup need to require that exact version. A user may (1) already have `foo` installed and then runtime fails (2) a different package may require a lower version of `foo` and thus `pip` won't upgrade it to the latest version available.

> For the log that is not displayed this is because no logger is set in `benchmark_test.py`. I will review that for later, thanks!

That was just an example, as I said most, if not all tests fail with the same cryptic error.

OK, I see better now. Thanks!!

I have fixed this in another PR by updating the piece of code you raised by an assert inside the `__init__` of the TFTrainer and now your example works fine. We do not want to fix the TensorFlow version for the entire lib but only for the trainer, at least for now.

I understand.

Please let me know which PR if you'd like me to re-test, or when it gets merged and I will re-test then.

Thank you, @jplu 

Now the fix is merged :)

Thank you for remembering to ping me. I re-tested with master and the tests now work with tensorflow==2.0.1 - thank you very much, @jplu
Transformer-XL: Fixed returned outputs when using `return_tuple=True`
Author: RafaelWO
Pull request: yes

Fixes #5974

`mems` are returned again when using `return_tuple=True`.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5975?src=pr&el=h1) Report
> Merging [#5975](https://codecov.io/gh/huggingface/transformers/pull/5975?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/ae67b2439fb15954bfd8f0fdf521cf1a650bafb9&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `0.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5975/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5975?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #5975   +/-   ##
=======================================
  Coverage   78.51%   78.51%           
=======================================
  Files         146      146           
  Lines       26214    26214           
=======================================
+ Hits        20581    20582    +1     
+ Misses       5633     5632    -1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5975?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_transfo\_xl.py](https://codecov.io/gh/huggingface/transformers/pull/5975/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190cmFuc2ZvX3hsLnB5) | `79.16% <0.00%> (ø)` | |
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5975/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `86.46% <0.00%> (+0.25%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5975?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5975?src=pr&el=footer). Last update [ae67b24...ed8722d](https://codecov.io/gh/huggingface/transformers/pull/5975?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Was fixed in different PR: #5999
Upgrade pip when doing CI
Author: LysandreJik
Pull request: yes

Try to fix the flaky tests by upgrading pip before installing dependencies.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6234?src=pr&el=h1) Report
> Merging [#6234](https://codecov.io/gh/huggingface/transformers/pull/6234?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/57eb1cb68d1c567b25ac256444e5c1a77b8817a7&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/6234/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/6234?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #6234   +/-   ##
=======================================
  Coverage   79.29%   79.29%           
=======================================
  Files         146      146           
  Lines       26597    26597           
=======================================
  Hits        21089    21089           
  Misses       5508     5508           
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/6234?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6234?src=pr&el=footer). Last update [57eb1cb...243c98f](https://codecov.io/gh/huggingface/transformers/pull/6234?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Make the order of additional special tokens deterministic
Author: gonglinyuan
Pull request: yes

In `SpecialTokensMixin.all_special_tokens_extended`, deduplication is performed by `all_toks = list(set(all_toks))`. However, this will change the ordering of additional special tokens, and the order depends on the hash seed of set data structure. This will result in non-deterministic id of additional special tokens added to `AutoTokenizer.from_pretrained` method. Therefore, I changed this problematic line to `all_toks = list(OrderedDict.fromkeys(all_toks))`. This line will deduplicate `all_toks` while still keeping the original ordering.


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5704?src=pr&el=h1) Report
> Merging [#5704](https://codecov.io/gh/huggingface/transformers/pull/5704?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/0befb513278f6e42b722be340dbc667e0ba2718e&el=desc) will **decrease** coverage by `0.94%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5704/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5704?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5704      +/-   ##
==========================================
- Coverage   78.26%   77.32%   -0.95%     
==========================================
  Files         146      146              
  Lines       25998    25998              
==========================================
- Hits        20348    20102     -246     
- Misses       5650     5896     +246     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5704?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/tokenization\_utils\_base.py](https://codecov.io/gh/huggingface/transformers/pull/5704/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdXRpbHNfYmFzZS5weQ==) | `93.60% <100.00%> (ø)` | |
| [src/transformers/modeling\_tf\_mobilebert.py](https://codecov.io/gh/huggingface/transformers/pull/5704/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9tb2JpbGViZXJ0LnB5) | `23.38% <0.00%> (-73.39%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_electra.py](https://codecov.io/gh/huggingface/transformers/pull/5704/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9lbGVjdHJhLnB5) | `95.53% <0.00%> (+69.51%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5704?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5704?src=pr&el=footer). Last update [0befb51...c706cc9](https://codecov.io/gh/huggingface/transformers/pull/5704?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Save model for tensorflow serving
Author: elixium
Pull request: no

Hello,
Thanks for the library. I tried your Multi label classification. I trained it with my data. It worked very accurate and fast. Now i want to use this model with tensorflow. I am new on pytorch and i looked some tutorials. As i understand i need to save model then convert to Onnx then to tensorflow. So I tried to save model first but it gave me this error;
```
AttributeError                            Traceback (most recent call last)
<ipython-input-10-4f1e1257e6e8> in <module>()
      1 
----> 2 torch.save(model.state_dict(), 'output/multilabel.pth')

AttributeError: 'MultiLabelClassificationModel' object has no attribute 'state_dict'
```

How can i save the model and export to onnx?

Comments:
Did you get the solution to this? @elixium 

Hi any update on this? I would like to deploy huggingface Transformers model with Tensorflow Serving too
Add license info to German Bert models
Author: Timoeller
Pull request: yes
Labels: name: model card


Comments:
Sorry, it is really annoying that it shows old commits (that have been squashed and merged into transformers in a previous PR).

Any idea how I could remove those old commits?

No big deal, I'll squash everything anyways.

In the future you can just delete your master branch and recreate it from upstream.
Discrepancy in the pad_token_id between the tokenizer and the model code of the T5
Author: eyal-str
Pull request: no

- `transformers` version: 2.11.0
- Platform: Darwin-19.3.0-x86_64-i386-64bit
- Python version: 3.7.6

 tokenizers: @mfuntowicz
 T5: @patrickvonplaten

It seems that there's a discrepancy between the tokenizer and the model code of the T5 regarding the pad_token_id.
looking at the following output the pad_token_id is 0:
```
from transformers import T5Tokenizer
tokenizer = T5Tokenizer.from_pretrained('t5-large')
tokenizer.pad_token_id
Out[4]: 0
```

When calculating the loss. I would expect the calculation to ignore the paddings, but when looking at the code, it ignores token_id = -100.
modeling_t5.py:
```
        if lm_labels is not None:
            loss_fct = CrossEntropyLoss(ignore_index=-100)
            loss = loss_fct(lm_logits.view(-1, lm_logits.size(-1)), lm_labels.view(-1))
            # TODO(thom): Add z_loss https://github.com/tensorflow/mesh/blob/fa19d69eafc9a482aff0b59ddd96b025c0cb207d/mesh_tensorflow/layers.py#L666
            decoder_outputs = (loss,) + decoder_outputs
```




Comments:
Hi @eyal-str , `pad_token_id` is indeed 0 for T5. But it's a convention to use -100 as ignore index in `CrossEntropyLoss`, so when preparing `labels` the `pad_token_ids` are replaced by -100. 

@patil-suraj Thanks for the prompt reply. I just saw what you meant in the DataCollatorForLanguageModeling.mask_tokens function.
TOKENIZER: truncation not working for batch
Author: PyAntony
Pull request: no

## Environment info

- `transformers` version: 2.11.0
- Platform: Linux-4.15.0-106-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.4
- PyTorch version (GPU?): 1.5.1 (False)
- Tensorflow version (GPU?): 2.2.0 (False)
- Using GPU in script?: NO
- Using distributed or parallel set-up in script?: NO

### Who can help
 
 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 examples/distillation: @VictorSanh

## Information

Model I am using: DistilBertForSequenceClassification.

The tokenizer does not truncate when I pass a list of strings. It only works when I pass a single string.

## To reproduce

Copy/paste (or just read) code below.  

```python
from transformers import DistilBertTokenizer

tokenizer = DistilBertTokenizer.from_pretrained("distilbert-base-uncased")

sentence = "Submit a bug report to help us improve transformers."
output = tokenizer(sentence, padding=True, truncation=True, max_length=4)

print(output['input_ids'])
# 4 tokens as expected from *max_length=4*
# out: [101, 12040, 1037, 102]

# now let's test with multiple sentences
sentences = [
  "Submit a bug report to help us improve transformers.", 
  "Benchmark a part of this library and share your results"
] 

output = tokenizer(sentences, padding=True, truncation=True, max_length=4)
print(output['input_ids'])
# output is returning all tokens, it is not truncating to max_length!
# out: [[101, 12040, 1037, 11829, 3189, 2000, 2393, 2149, 5335, 19081, 1012, 102, 0], 
#       [101, 6847, 10665, 1037, 2112, 1997, 2023, 3075, 1998, 3745, 2115, 3463, 102]]
```

## Expected 
```python
# output truncated to max_length (4 as in the example)
# out: [[101, 12040, 1037, 102], 
#       [101, 6847, 10665, 102]]
```


Comments:
I have updated to version 3.0.2 now and the tokenizer is working properly. I am closing this issue.
Create README.md
Author: mrm8488
Pull request: yes
Labels: name: model card


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6075?src=pr&el=h1) Report
> Merging [#6075](https://codecov.io/gh/huggingface/transformers/pull/6075?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/9d0d3a6645384e236c55d311f3f8b7dd67d58562&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/6075/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/6075?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #6075   +/-   ##
=======================================
  Coverage   78.59%   78.59%           
=======================================
  Files         146      146           
  Lines       26314    26314           
=======================================
  Hits        20681    20681           
  Misses       5633     5633           
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/6075?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6075/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `82.20% <0.00%> (-0.29%)` | :arrow_down: |
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6075/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `86.46% <0.00%> (+0.25%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/6075?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6075?src=pr&el=footer). Last update [9d0d3a6...de221b0](https://codecov.io/gh/huggingface/transformers/pull/6075?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


What's the difference between this and `_v2`?

v2 includes ingredients too. V1 just includes instructions (the dataset for this version was provided by https://twitter.com/alexcg/status/1286464335867346946?s=19 But then I found a version with ingredients and instructions and created v2.

👍
Update model card
Author: alisafaya
Pull request: yes
Labels: name: model card


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6198?src=pr&el=h1) Report
> Merging [#6198](https://codecov.io/gh/huggingface/transformers/pull/6198?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/82a0e2b67ec94d28b20e24b3393644002bbd0d4b&el=desc) will **decrease** coverage by `0.80%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/6198/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/6198?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #6198      +/-   ##
==========================================
- Coverage   79.65%   78.85%   -0.81%     
==========================================
  Files         146      146              
  Lines       26607    26607              
==========================================
- Hits        21194    20981     -213     
- Misses       5413     5626     +213     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/6198?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/tokenization\_xlm.py](https://codecov.io/gh/huggingface/transformers/pull/6198/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25feGxtLnB5) | `16.26% <0.00%> (-66.67%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_distilbert.py](https://codecov.io/gh/huggingface/transformers/pull/6198/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9kaXN0aWxiZXJ0LnB5) | `64.18% <0.00%> (-34.62%)` | :arrow_down: |
| [src/transformers/tokenization\_marian.py](https://codecov.io/gh/huggingface/transformers/pull/6198/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fbWFyaWFuLnB5) | `68.14% <0.00%> (-25.67%)` | :arrow_down: |
| [src/transformers/tokenization\_bart.py](https://codecov.io/gh/huggingface/transformers/pull/6198/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fYmFydC5weQ==) | `95.77% <0.00%> (+35.21%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_flaubert.py](https://codecov.io/gh/huggingface/transformers/pull/6198/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9mbGF1YmVydC5weQ==) | `88.19% <0.00%> (+63.97%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/6198?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6198?src=pr&el=footer). Last update [82a0e2b...937eea4](https://codecov.io/gh/huggingface/transformers/pull/6198?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
add gpt2 padding for tflite
Author: gyin94
Pull request: no

# ❓ Questions & Help

Since tflite requires a fixed input length, how can we correctly add padding for gpt2 input with a fix length so that it won't affect the final result? 

```
import tensorflow as tf
from transformers import *
gpt2_model = TFGPT2LMHeadModel.from_pretrained('distilgpt2')
class WrapModel(tf.keras.models.Model):
    def __init__(self, transformer):
        super(WrapModel, self).__init__()
        self.transformer = transformer

    def call(self, input_ids, **kwargs):
        inputs = {"inputs": input_ids}
        outputs = self.transformer(**inputs)
        next_token_logits = outputs[0][:, -1, :]

        # Greedy decoding
        next_token = tf.math.argmax(next_token_logits, axis=-1, output_type=tf.int32)
        return {"decoded_ids": next_token}

w = WrapModel(gpt2_model)
input_layer = tf.keras.layers.Input(shape=(1, 10), dtype=tf.int32, name='input_ids')
prediction_model = w(input_layer)
tf_model = tf.keras.models.Model(inputs=input_layer, outputs=prediction_model)
export_dir = "./gpt2_tmp/"
tf.keras.models.save_model(tf_model, export_dir)

saved_model_dir = "./gpt2_tmp/"
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
converter.experimental_new_converter = True
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_quant_model = converter.convert()
```

I am not clear how we can use attention_mask to solve this concern based on this one. https://github.com/huggingface/transformers/issues/2630

Comments:
the latest TF2.3 and tflite can support dynamic input now. Close this.