noamaanMulla-03/transformers-issues huggingface dataset:
DistilBert not giving hidden states
Author: ierezell
Pull request: no

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): DistilBert

Language I am using the model on (English, Chinese ...): Multilingual

The problem arises when using:
* [X] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name) just running inference
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Load modules
2. Create model with output_hidden_layers=True
3. run inference on one sentece

`from transformers import BertModel, BertTokenizerFast, BertConfig`
`from transformers import DistilBertModel, DistilBertTokenizerFast, DistilBertConfig`
`import torch`

`bert_config = BertConfig(output_hidden_states=True)`
`bert = BertModel(bert_config)`
`bert = bert.from_pretrained("bert-base-multilingual-cased")`
`bert_tokenizer = BertTokenizerFast.from_pretrained("bert-base-multilingual-cased")`


`distil_bert_config = DistilBertConfig(output_hidden_states=True)`
`distil_bert = DistilBertModel(distil_bert_config)`
`distil_bert = distil_bert.from_pretrained("distilbert-base-multilingual-cased")`
`distil_bert_tokenizer = DistilBertTokenizerFast.from_pretrained("distilbert-base-multilingual-cased")`

`sentence = "One stupid dummy sentence to test !"`

`input_bert = torch.tensor(bert_tokenizer.encode(sentence)).unsqueeze(0)`
`input_distil_bert = torch.tensor(distil_bert_tokenizer.encode(sentence)).unsqueeze(0)
`

`output_bert = bert(input_bert)`
`outupt_distil_bert = distil_bert(input_distil_bert)`
## Expected behavior
Return a tuple with 2 elements (like Bert)

Exemple of bert and desired behavior of distilbert: 
`print(len(output_bert)) # => 2`
`print(output_bert[0].size()) # => 1, 18, 768`
`print(output_bert[1].size()) # => 1, 768`

## Real behavior 
Return tuple of only 1 element 
`print(len(outupt_distil_bert)) # => 1`
`print(outupt_distil_bert[0].size()) # => 1, 18, 768`

## Environment info
- `transformers` version: 2.7.0
- Platform: Linux 5.5.8-arch1-1
- Python version: Python 3.8.1
- PyTorch version (GPU?): 1.4.0 (GPU)
- Tensorflow version (GPU?): None
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


Comments:
@Ierezell I am facing same issue. How did you fix this issue?
uncased readme
Author: kldarek
Pull request: yes
Labels: name: model card

updates to the model card for uncased model with more evaluation results and recommendation to switch to cased model

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4608?src=pr&el=h1) Report
> Merging [#4608](https://codecov.io/gh/huggingface/transformers/pull/4608?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/a9aa7456ac824c9027385b149f405e4f5649273f&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4608/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4608?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4608   +/-   ##
=======================================
  Coverage   78.02%   78.02%           
=======================================
  Files         124      124           
  Lines       20626    20626           
=======================================
+ Hits        16093    16094    +1     
+ Misses       4533     4532    -1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4608?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4608/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.85% <0.00%> (+0.41%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4608?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4608?src=pr&el=footer). Last update [a9aa745...470e98f](https://codecov.io/gh/huggingface/transformers/pull/4608?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Removed the use of deprecated Variable API in PPLM example
Author: prajjwal1
Pull request: yes


Comments:
Could you please tell what does this mean ?
```
#!/bin/bash -eo pipefail
black --check --line-length 119 --target-version py35 examples templates tests src utils

would reformat /home/circleci/transformers/examples/pplm/run_pplm.py
Oh no! 💥 💔 💥
1 file would be reformatted, 266 files would be left unchanged.

Exited with code exit status 1
```

You need to run `make style` and `make quality` locally and push the resulting changes

Also this file was moved around so you'll need to re-apply your changes to new location

I ran both make commands as you said.
```
(base) u37216@s001-n007:~/transformers$ make style
black --line-length 119 --target-version py35 examples templates tests src utils
reformatted /home/u37216/transformers/src/transformers/__init__.py
reformatted /home/u37216/transformers/templates/adding_a_new_example_script/run_xxx.py
reformatted /home/u37216/transformers/templates/adding_a_new_example_script/utils_xxx.py
All done! ✨ 🍰 ✨
3 files reformatted, 272 files left unchanged.
isort --recursive examples templates tests src utils
Fixing /home/u37216/transformers/templates/adding_a_new_example_script/run_xxx.py
Fixing /home/u37216/transformers/templates/adding_a_new_example_script/utils_xxx.py
Fixing /home/u37216/transformers/src/transformers/__init__.py
(base) u37216@s001-n007:~/transformers$ make quality
black --check --line-length 119 --target-version py35 examples templates tests src utils
would reformat /home/u37216/transformers/src/transformers/__init__.py
would reformat /home/u37216/transformers/templates/adding_a_new_example_script/run_xxx.py
would reformat /home/u37216/transformers/templates/adding_a_new_example_script/utils_xxx.py
Oh no! 💥 💔 💥
3 files would be reformatted, 272 files would be left unchanged.
Makefile:6: recipe for target 'quality' failed
make: *** [quality] Error 1
```
Is this related to my code ? I didn't even touch other parts of the file. I think the formatting issue came from master itself (I checked it). 

@julien-c Can you please see this ?
README for HooshvareLab
Author: m3hrdadfi
Pull request: yes
Labels: name: model card

HooshvareLab/bert-base-parsbert-uncased

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4610?src=pr&el=h1) Report
> Merging [#4610](https://codecov.io/gh/huggingface/transformers/pull/4610?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/a9aa7456ac824c9027385b149f405e4f5649273f&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4610/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4610?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4610   +/-   ##
=======================================
  Coverage   78.02%   78.02%           
=======================================
  Files         124      124           
  Lines       20626    20626           
=======================================
+ Hits        16093    16094    +1     
+ Misses       4533     4532    -1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4610?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4610/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `88.66% <0.00%> (+0.16%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4610?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4610?src=pr&el=footer). Last update [a9aa745...3db079d](https://codecov.io/gh/huggingface/transformers/pull/4610?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
[Model Card] model card for longformer-base-4096-finetuned-squadv1
Author: patil-suraj
Pull request: yes
Labels: name: model card

@patrickvonplaten 

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4625?src=pr&el=h1) Report
> Merging [#4625](https://codecov.io/gh/huggingface/transformers/pull/4625?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/6a17688021268fe429e78c66ea0932cb55cd03b1&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4625/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4625?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4625      +/-   ##
==========================================
- Coverage   78.02%   78.00%   -0.02%     
==========================================
  Files         124      124              
  Lines       20635    20635              
==========================================
- Hits        16100    16097       -3     
- Misses       4535     4538       +3     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4625?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4625/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.44% <0.00%> (-0.42%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4625/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `88.50% <0.00%> (-0.17%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4625/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `90.41% <0.00%> (-0.12%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4625?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4625?src=pr&el=footer). Last update [6a17688...8c230e9](https://codecov.io/gh/huggingface/transformers/pull/4625?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


That's great thanks @patil-suraj
question about 'add_prefix_space' of encode method
Author: weiguowilliam
Pull request: no

## ❓ Questions & Help

<!-- A clear and concise description of the question. -->

Hi, I have a question about the 'add_prefix_space' parameter of encode method. I'll use small gpt2 for example.

> tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
> print(tokenizer.encode("my",add_prefix_space=True)) #616
> print(tokenizer.encode("my",add_prefix_space=False)) #1820

I think 616 is the index for the word 'my'(' my') since there's a space before letter 'm'. 
And 1820 is the index for '-me' which is part of a word. Am I right? 

Thank you!

Comments:
Actually, both 616 and 1820 are indices for `my`. The difference lies in the prefix space that was added: in the case that it was not added (1820), it is identified as being the beginning of the sentence or part of a word.

In the case that it was added (616), it is identified as being the beginning of a word in a sentence.

You can check the behavior by calling `tokenize` instead of `encode`, for example:

```py
tokenizer = GPT2Tokenizer.from_pretrained('gpt2')

print(tokenizer.tokenize("my son is jeremy",add_prefix_space=True))
# ['Ġmy', 'Ġson', 'Ġis', 'Ġj', 'ere', 'my']

print(tokenizer.tokenize("my son is my son",add_prefix_space=False))
# ['my', 'Ġson', 'Ġis', 'Ġmy', 'Ġson']
```

Thank you! 

The argument you mention is for the initialization of a tokenizer from the `huggingface/tokenizers` library, you would have better luck opening an issue there.
[examples] fix summarization do_predict
Author: sshleifer
Pull request: yes

- by copying NER
- add type hints

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3866?src=pr&el=h1) Report
> Merging [#3866](https://codecov.io/gh/huggingface/transformers/pull/3866?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/a21d4fa410dc3b4c62f93aa0e6bbe4b75a101ee9&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3866/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3866?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #3866   +/-   ##
=======================================
  Coverage   78.61%   78.62%           
=======================================
  Files         106      106           
  Lines       17953    17953           
=======================================
+ Hits        14114    14115    +1     
+ Misses       3839     3838    -1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3866?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3866/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.06% <0.00%> (+0.12%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3866?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3866?src=pr&el=footer). Last update [a21d4fa...c84984c](https://codecov.io/gh/huggingface/transformers/pull/3866?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
NER Pipeline Issue
Author: ElToro13
Pull request: no
Labels: name: Core: Pipeline

name: Ex: Named Entity Recognition

I am trying to run NER Pipeline. Here, I am using the line, "Statue of Liberty is located in New York". I am getting the following output
[{'entity': 'I-MISC', 'score': 0.5469961762428284, 'word': 'St'},
 {'entity': 'I-MISC', 'score': 0.7588933706283569, 'word': '##at'},
 {'entity': 'I-MISC', 'score': 0.5194069147109985, 'word': '##ue'},
 {'entity': 'I-MISC', 'score': 0.8465802073478699, 'word': 'of'},
 {'entity': 'I-PER', 'score': 0.4912404716014862, 'word': 'Liberty'},
 {'entity': 'I-LOC', 'score': 0.9995675086975098, 'word': 'New'},
 {'entity': 'I-LOC', 'score': 0.999152660369873, 'word': 'York'}]

My Issue is, why is it breaking down to individual words. Is there a way to chunk?

Comments:
I am suffering from the same problem, trying to recover input texts from `examples/run_ner.py`.

Agree. I find the use of BIO very unorthodox in this case; if B actually represented the beginning of an entity (vs. the beginning of the new entity of the same type), we could reconstruct these spans ourselves. Currently I don't think it's possible to perfectly reconstruct them, though.

I believe this issue should be resolved by this recently merged [PR](https://github.com/huggingface/transformers/pull/3957), which allows for the extraction of **entity groups** 🙂 

Indeed, thanks @enzoampil!
When i fine tune the BERT on my serve, it always says Segmentation fault? 
Author: bianximo
Pull request: no
Labels: name: Need more information

name: wontfix

I have done some modification on BertForSequenceClassification to apply a mutilabel prediction task, but
when i run my code on my serve, it always says that Segmentation fault when it runs to 
"loss = model(input_ids, segment_ids, input_mask,label_ids)", and even if i don't use GPU and fp16, it comes the same fault.
but if i run the code on my PC ,it works well,just very slow....
what 's wrong??

Comments:
I had the same issue. I'm getting seg fault on an aws deep learning ami with a tesla v100 gpu instance. I get the error with or with out fp16

I also have the same problem. Did you guys figure out any solution? 
I am able to load the data, however at the first epoch 0, I see the error segmentation fault. 

Can you try running the model:
- with a very small batch size to check if it's an OOM error
- with `CUDA_LAUNCH_BLOCKING=1` to see the exact line causing the error
?

setting the env variable CUDA_LAUNCH_BLOCKING=1 doesn't give me any additional error messages. I still see the same error. 
 
Tried it with batch size 1 and I get the same issue

I got the code to run on my school server. I updated the gcc using this `conda install -c psi4 gcc-5`. 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


I am also experiencing this issue. Anyone else figure it out?
Onnx notebook problem
Author: amy-hyunji
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->
I saw the issue related to mine in #260 so I changed env to python 3.6 and torch 1.1 but I didn't help.
When I run onnx notebook I get an error [TypeError: export() got an unexpected keyword argument 'dynamic_axes']
Does anyone have guess what's wrong?
<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


Comments:
You can try newer version of PyTorch (like 1.3~1.5). The problem shall be resolved.

Hi thanks for the reply
I tried with pytorch 1.4 but I got another error on the below
![image](https://user-images.githubusercontent.com/44370759/83090959-fad51180-a0d4-11ea-8db7-53b96e20e0c3.png)
do you have any idea about this one?
thanks!


@amy-hyunji, this option (use_external_data_format) need PyTorch 1.5. This option is not needed for model < 2GB.

If you do not want to upgrade to PyTorch 1.5. You can install transformers from source, and modify the convert_graph_to_onnx.py  (by removing the parameter during calling onnx.export function).

@tianleiwu Thanks a lot :)
How to speed up inference step in BertQuestionAnswering?
Author: minhnguyenth
Pull request: no
Labels: name: External

name: Distillation

I'm working on a QA system which uses a pre-trained BertQA model. At this point, even if I use GPU, the step generating start_scores and end_scores  for a set of 20 candidate passages till takes a few seconds which is the bottleneck of my application.

I just wonder do we have any strategies/tricks to speed up this step? So far, it seems using multiple GPUs at the inference step does not help at all. Any advice greatly appreciated!

Comments:
To improve inference speed you can use ONNX (also see here: https://github.com/huggingface/transformers/issues/260). In addition, you can opt for a distilled model rather than the full model. 

Hi! I've used this:


distilbert-base-uncased-distilled-squad |  
-- | --
or 

distilbert-base-cased-distilled-squad |  
-- | --

It improved quite a bit! 


@ZordoC how much speed improvement did you observe?
[Contributing Doc] Update version command when contributing
Author: patrickvonplaten
Pull request: yes

According to PR: #4131, the `CONTRIBUTING.md` should be updated a bit. @BramVanroy 

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4614?src=pr&el=h1) Report
> Merging [#4614](https://codecov.io/gh/huggingface/transformers/pull/4614?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/a9aa7456ac824c9027385b149f405e4f5649273f&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4614/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4614?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4614   +/-   ##
=======================================
  Coverage   78.02%   78.03%           
=======================================
  Files         124      124           
  Lines       20626    20626           
=======================================
+ Hits        16093    16095    +2     
+ Misses       4533     4531    -2     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4614?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4614/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `90.53% <0.00%> (+0.11%)` | :arrow_up: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4614/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.85% <0.00%> (+0.41%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4614?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4614?src=pr&el=footer). Last update [a9aa745...cbab365](https://codecov.io/gh/huggingface/transformers/pull/4614?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Good catch, thanks!