noamaanMulla-03/transformers-issues huggingface dataset:
considering empty input text to avoid string index out of range error
Author: hmohebbi
Pull request: yes
Labels: name: wontfix

When the input string is empty, the above condition in the tokenization_roberta will encounter string index out of range error. For example, a pair input string exists in the QQP dataset that has an empty string:

{'idx': 362246,
 'label': 0,
 'question1': b'How can I develop android app?',
 'question2': b''}

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Fix pytorch lighting examples
Author: simonepri
Pull request: yes
Labels: name: wontfix

It throws the exception 'Trainer' object has no attribute 'avg_loss' because since version 0.7.2 they removed the avg_loss field from the Trainer class.

Also `get_tqdm_dict` is deprecated since 0.7.3.

See https://github.com/huggingface/transformers/pull/2890#issuecomment-613066707

Comments:
@simonepri can you point to the examples that need fixing?

https://github.com/huggingface/transformers/blob/master/examples/glue/run_pl_glue.py
https://github.com/huggingface/transformers/blob/master/examples/ner/run_pl_ner.py


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Feature/torchserve interface [WIP]
Author: MFreidank
Pull request: yes
Labels: name: wontfix

Work in progress PR to add a CLI interface for easy packaging of transformers models 
for serving in [pytorch/serve](https://github.com/pytorch/serve). 

Intended usage example:
`transformers-cli torchserve --checkpoint="distilbert-base-uncased-finetuned-sst-2-english" --tokenizer="distilbert-base-uncased" --model-name="distilbert" --task="sentiment-analysis"`

The call above produces a MAR (model archive) file that can be served directly by the `torchserve` binary. 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Fix object is not subscriptable error in BertEncoder (#1188)
Author: mklimasz
Pull request: yes
Labels: name: wontfix

Fix object is not subscriptable error in BertEncoder when head mask is None.

Issue #1188 describes problem.

BertLayer accepts head_mask as None, however if BertEncoder gets head_mask as None - it tries to index None.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4095?src=pr&el=h1) Report
> Merging [#4095](https://codecov.io/gh/huggingface/transformers/pull/4095?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/b8686174be75220d2c26a961597a39ef4921b616&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `50.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4095/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4095?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4095   +/-   ##
=======================================
  Coverage   78.84%   78.84%           
=======================================
  Files         114      114           
  Lines       18691    18693    +2     
=======================================
+ Hits        14737    14739    +2     
  Misses       3954     3954           
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4095?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_bert.py](https://codecov.io/gh/huggingface/transformers/pull/4095/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19iZXJ0LnB5) | `88.68% <50.00%> (-0.15%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4095/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `92.61% <0.00%> (-0.17%)` | :arrow_down: |
| [src/transformers/trainer.py](https://codecov.io/gh/huggingface/transformers/pull/4095/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmVyLnB5) | `43.90% <0.00%> (+0.34%)` | :arrow_up: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4095/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.55% <0.00%> (+0.41%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4095?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4095?src=pr&el=footer). Last update [b868617...d506143](https://codecov.io/gh/huggingface/transformers/pull/4095?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Indeed :)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Delete batch = tuple(t.to(args.device) for t in batch) for it perform…
Author: DrJZhou
Pull request: yes
Labels: name: wontfix

…s two times

Comments:
This seems reasonable, what do you think @suvrat96?

(never mind the failing CI test)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
info about loading file None is not informative
Author: max-yue
Pull request: yes
Labels: name: wontfix

When file_path is None do not say loading file None

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4042?src=pr&el=h1) Report
> Merging [#4042](https://codecov.io/gh/huggingface/transformers/pull/4042?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/180585741cf3cdd6890cb99610923a8ae9691220&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4042/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4042?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4042   +/-   ##
=======================================
  Coverage   78.50%   78.50%           
=======================================
  Files         111      111           
  Lines       18492    18492           
=======================================
+ Hits        14517    14518    +1     
+ Misses       3975     3974    -1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4042?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/tokenization\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4042/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdXRpbHMucHk=) | `89.72% <100.00%> (ø)` | |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4042/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `72.61% <0.00%> (-0.42%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4042/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.06% <0.00%> (+0.12%)` | :arrow_up: |
| [src/transformers/trainer.py](https://codecov.io/gh/huggingface/transformers/pull/4042/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmVyLnB5) | `43.90% <0.00%> (+0.34%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4042?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4042?src=pr&el=footer). Last update [1805857...789019b](https://codecov.io/gh/huggingface/transformers/pull/4042?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
override weights name
Author: ofrik
Pull request: yes
Labels: name: wontfix

Optional override of the weights name, relevant for cases where torch.save() is patched like in https://github.com/allegroai/trains 

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3969?src=pr&el=h1) Report
> Merging [#3969](https://codecov.io/gh/huggingface/transformers/pull/3969?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/4e817ff41885063e08bb3bcd63e5adfd835b9911&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `82.60%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3969/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3969?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3969      +/-   ##
==========================================
+ Coverage   78.44%   78.45%   +0.01%     
==========================================
  Files         111      111              
  Lines       18518    18527       +9     
==========================================
+ Hits        14527    14536       +9     
  Misses       3991     3991              
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3969?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3969/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `92.81% <80.00%> (+0.04%)` | :arrow_up: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3969/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.11% <84.61%> (+0.17%)` | :arrow_up: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3969/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `68.49% <0.00%> (-0.37%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3969?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3969?src=pr&el=footer). Last update [4e817ff...56024ae](https://codecov.io/gh/huggingface/transformers/pull/3969?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
[examples] fine-tuning `bert-base-finnish-(un)cased-v1` model for Named Entity Recognition
Author: bmichele
Pull request: yes
Labels: name: wontfix

No new features added and no modifications to the existing code has been done. Just addition of scripts for the fine-tuning.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3474?src=pr&el=h1) Report
> Merging [#3474](https://codecov.io/gh/huggingface/transformers/pull/3474?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/e392ba6938f50655a195ea7ec8a260b1e9fc6058&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3474/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3474?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #3474   +/-   ##
=======================================
  Coverage   77.56%   77.56%           
=======================================
  Files         100      100           
  Lines       16970    16970           
=======================================
+ Hits        13162    13163    +1     
+ Misses       3808     3807    -1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3474?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3474/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.85% <0.00%> (+0.13%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3474?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3474?src=pr&el=footer). Last update [e392ba6...d9e6d4e](https://codecov.io/gh/huggingface/transformers/pull/3474?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Optimize tokenization (85-92% time reduction)
Author: soni-n
Pull request: yes
Labels: name: wontfix

This optimization in tokenization reduces the tokenization time by ~ 85-92 %

Stats:

Before fix: 
 8%|████████▉                                                                                                      | 119574/1486030 [01:03<12:09, 1874.22it/s]

After fix: 
 98%|████████████████████████████████████  | 1459590/1486030 [01:59<00:02, 12244.82it/s]


Another stats: 
When huge number of new tokens (~58K) are added, tokenization time bumps up to ~27-28 hours 
With this fix the time drops to ~2.15 hours !

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
add output_past option to BERT class 
Author: asahi417
Pull request: yes
Labels: name: wontfix

I need key-value of the present state like GPT class for BERT as well (I'm testing PPLM like architecture but with masked LM), and here I've added the option `output_past` to BERT to enable returning those statistics.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Directories not found when saving checkpoints
Author: nguyenhoan1988
Pull request: yes
Labels: name: wontfix

_rotate_checkpoints deletes some checkpoint directory which causes "directories not found" error

Comments:
Do you mind making sure the code quality test runs before we merge? You can see how to do that in the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Only self-attention returned when output_attentions = True for BERT
Author: YovaKem
Pull request: no
Labels: name: wontfix

# 🐛 Bug
```
if self.output_attentions:
                all_attentions = all_attentions + (layer_outputs[1],)
```
should be
```
if self.output_attentions:
                all_attentions = all_attentions + (layer_outputs[1:],)
```
## Information
Line 412 [here](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bert.py
)

I assume that there is no reason to exclude cross-attention from all_attentions. 


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
T5 ONNX Export Test Failing on GPU
Author: sshleifer
Pull request: no

https://github.com/huggingface/transformers/runs/863647649?check_suite_focus=true

![image](https://user-images.githubusercontent.com/6045025/87354370-788d9900-c52c-11ea-8838-286633cd93dc.png)


Comments:
[bart] decoder.last_hidden_state shape changes when passing labels
Author: sshleifer
Pull request: no

```
config = BartConfig(
            vocab_size=99,
            d_model=24,
            encoder_layers=2,
            decoder_layers=2,
            encoder_attention_heads=2,
            decoder_attention_heads=2,
            encoder_ffn_dim=32,
            decoder_ffn_dim=32,
            max_position_embeddings=48,
            add_final_layer_norm=True,
        )
        lm_model = BartForConditionalGeneration(config).to(torch_device)
        context = torch.Tensor([[71, 82, 18, 33, 46, 91, 2], [68, 34, 26, 58, 30, 2, 1]]).long().to(torch_device)
        summary = torch.Tensor([[82, 71, 82, 18, 2], [58, 68, 2, 1, 1]]).long().to(torch_device)
        loss, logits, enc_features = lm_model(input_ids=context, decoder_input_ids=summary, labels=summary)

        expected_shape = (*summary.shape, config.vocab_size)
        self.assertEqual(logits.shape, expected_shape)
        outputs2 = lm_model(input_ids=context, decoder_input_ids=summary)
        self.assertEqual(outputs2.logits.shape, expected_shape) 
# Fails torch.Size([2, 1, 99]) != (2, 5, 99)
```
Is this expected @sgugger ?

Comments:
NVM, it's a me problem.

FFR, You have to manually pass `use_cache=False` to bart forward now if you're not generating
Sentence-transformers model outputs different than when loaded in HuggingFace
Author: petulla
Pull request: no

# 🐛 Bug

## Information

The outputs I'm seeing from generating sentence embeddings with the steps in the model zoo for getting sentence-transformers embeddings deviate from those generated from the SentenceTransformers package. 

I'm assuming this is due to the lack of pooling. Is there a way to convert a ST model to HF with pooling?

English

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Load a pre-trained model from SentenceTransformers
2. Generate sentence embeddings from tokenized inputs
3. Print sentence embeddings

HuggingFace:
```python
#Sentences we want sentence embeddings for
sentences = ['This framework generates embeddings for each input sentence',
             'Sentences are passed as a list of string.',
             'The quick brown fox jumps over the lazy dog.']
tokenizer = AutoTokenizer.from_pretrained("roberta-large-nli-stsb-mean-tokens/0_RoBERTa/")
>>> model = AutoModel.from_pretrained("roberta-large-nli-stsb-mean-tokens/0_RoBERTa/")
>>> encoded_input = tokenizer(sentences, padding=True, truncation=True, max_length=128, return_tensors='pt')
>>> with torch.no_grad():
...     model_output = model(**encoded_input)
...     sentence_embeddings = model_output[0][:,0]
...
>>> print("Sentence embeddings:")
Sentence embeddings:
>>> print(sentence_embeddings)
tensor([[ 0.0057, -0.7690,  0.0702,  ...,  0.0734, -1.4343,  0.3418],
        [ 0.2066, -0.8213,  0.1272,  ...,  0.2649, -1.2799, -0.1636],
        [-0.4860, -0.5176, -0.5924,  ..., -0.4880, -0.1880, -0.0554]])
```

SentenceTransformers

```python
model = SentenceTransformer('roberta-large-nli-stsb-mean-tokens')
sentence_embeddings = model.encode(sentences)>>> sentence_embeddings = model.encode(sentences)
>>> sentence_embeddings = model.encode(sentences)
>>> print(sentence_embeddings)
[array([ 0.6306487 , -0.2879937 ,  0.05334993, ...,  0.26865923,
       -2.2382815 ,  0.22505784], dtype=float32), array([ 0.22068763, -0.8045991 ,  0.18439776, ...,  0.6993382 ,
       -1.7670776 ,  0.11258417], dtype=float32), array([-0.17819108,  0.08762542, -0.7614953 , ..., -0.6983883 ,
       -0.13175072, -0.11123852], dtype=float32)]
```

## Expected behavior

SentenceTransformer and HF outputs should be the same

## Environment info

```
- `transformers` version: 3.0.2
- Platform: Darwin-18.7.0-x86_64-i386-64bit
- Python version: 3.7.5
- PyTorch version (GPU?): 1.5.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```


Comments:
This is just one of those days.. There's a[ mean pooling function](https://huggingface.co/sentence-transformers/bert-base-nli-mean-tokens) here that can be adapted.
metadata
Author: piegu
Pull request: yes
Labels: name: model card


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5758?src=pr&el=h1) Report
> Merging [#5758](https://codecov.io/gh/huggingface/transformers/pull/5758?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/8ab565a4be5a7fd96b19ef88d474037ef31f27e5&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5758/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5758?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #5758   +/-   ##
=======================================
  Coverage   77.32%   77.32%           
=======================================
  Files         146      146           
  Lines       26047    26047           
=======================================
  Hits        20141    20141           
  Misses       5906     5906           
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5758?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5758?src=pr&el=footer). Last update [8ab565a...a67f412](https://codecov.io/gh/huggingface/transformers/pull/5758?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
T5 fine-tuned model doesn't appear in the model hub
Author: mrm8488
Pull request: no

Hi guys,
I have fine-tuned T5-base on wikiSQL dataset and uploaded it to HF model hub. The problem is that the model doesn't appear there. AFAIK it is because the config file is not right. Should I change anything?

Thanks.

Comments:
I can see it now: https://huggingface.co/mrm8488/t5-base-finetuned-wikiSQL

Let us know if something looks wrong (otherwise, feel free to close)

Everything right. Maybe I searched it too soon and was not indexed yet.