noamaanMulla-03/transformers-issues huggingface dataset:
New model request: MobileBERT from Google 
Author: forresti
Pull request: no
Labels: name: wontfix

name: New model

# 🌟 New model addition
This issue is to request adding the MobileBERT model, which was recently released by Carnegie Mellon University, Google Research, and Google Brain.

## Model description
MobileBERT is a more computationally-efficient model for achieving BERT-base level accuracy on a smartphone.

<!-- Important information -->

## Open source status

* [x] the model implementation is available: https://github.com/google-research/google-research/tree/master/mobilebert
* [x] the model weights are available: [uncased_L-24_H-128_B-512_A-4_F-4_OPT](https://storage.googleapis.com/cloud-tpu-checkpoints/mobilebert/uncased_L-24_H-128_B-512_A-4_F-4_OPT.tar.gz) (MobileBERT Optimized Uncased English)
* [x] who are the authors: Zhiqing Sun, Hongkun Yu (@saberkun), Xiaodan Song, Renjie Liu, Yiming Yang, Denny Zhou

Comments:
https://github.com/lonePatient/MobileBert_PyTorch

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This was completed in #4901
Potential security vulnerability regarding Hosted Interface API?
Author: KerenzaDoxolodeo
Pull request: no

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): distilbert-base-cased-distilled-squad with huggingface.co's API

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: Question and answering

## To reproduce

Steps to reproduce the behavior:

Run this code on a notebook
```
answer = requests.post('https://api-inference.huggingface.co/models/distilbert-base-cased-distilled-squad', 
                       headers = {"Content-Type": "application/json"},
                       data = {"question":'What is my name?' ,
                              "context" :"My name is Batik"})
```

Result : 500 Error . With the following content when I executed `print(answer.content)`

```
b'{"error":"JSONDecodeError(\'Expecting value: line 1 column 1 (char 0)\')","traceback":"  File \\"/home/hf/api-inference/server.py\\", line 251, in model_forward\\n    inputs = await request.json()\\n  File \\"/home/hf/api-inference/.env/lib/python3.8/site-packages/starlette/requests.py\\", line 227, in json\\n    self._json = json.loads(body)\\n  File \\"/usr/lib/python3.8/json/__init__.py\\", line 357, in loads\\n    return _default_decoder.decode(s)\\n  File \\"/usr/lib/python3.8/json/decoder.py\\", line 337, in decode\\n    obj, end = self.raw_decode(s, idx=_w(s, 0).end())\\n  File \\"/usr/lib/python3.8/json/decoder.py\\", line 355, in raw_decode\\n    raise JSONDecodeError(\\"Expecting value\\", s, err.value) from None\\n"}'
```

## Expected behavior

I imagine we don't want to give the Error traceback for security reasons

## Environment info

I tried using Python's request and Postman and have the same outcome. Furthermore, I'm wondering why I have a Error 500.  Was my code wrong?


Comments:
Your `requests` syntax is incorrect, you're not actually sending the request body as json.

This works:
```python
answer = requests.post(
    "https://api-inference.huggingface.co/models/distilbert-base-cased-distilled-squad",
    json={"question": "What is my name?", "context": "My name is Batik"},
)
```

Regarding the stack trace, I do not see an issue with returning it (it's json-encoded), happy to discuss more.
How to perform Agglomerative clustering on Albert model last hidden state output, it gives tuple of tensors for each sequence in a sentence. I am processing multiple sentences from a document, How to achieve this ?
Author: daraksha-shirin
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
QA Pipeline: Key Error due to predicting a token outside of allowed context
Author: tholor
Pull request: no

# 🐛 Bug

## Information

Model: distilbert
Language: English
The problem arises when using: QA inference via `pipeline`

The pipeline throws an exception when the model predicts a token that is not part of the document (e.g. final special token).
In the example below, the model predicts token 13 to be the end of the answer span. 
The context however ends at token 12 and token 13 is the final [SEP] token. Therefore, we get a key error when trying to access 
`feature.token_to_orig_map[13])` in here: 

https://github.com/huggingface/transformers/blob/ce374ba87767d551f720242d5e64bfa976531079/src/transformers/pipelines.py#L1370-L1380

## To reproduce

```
nlp = pipeline("question-answering",model="distilbert-base-uncased-distilled-squad",
                                  tokenizer="distilbert-base-uncased",
                                  device=-1)

nlp(question="test finding", context="My name is Carla and I live in Berlin")
```

results in 

```
Traceback (most recent call last):
  File "/home/mp/deepset/dev/haystack/debug.py", line 16, in <module>
    nlp(question="test finding", context="My name is Carla and I live in Berlin")
  File "/home/mp/miniconda3/envs/py37/lib/python3.7/site-packages/transformers/pipelines.py", line 1316, in __call__
    for s, e, score in zip(starts, ends, scores)
  File "/home/mp/miniconda3/envs/py37/lib/python3.7/site-packages/transformers/pipelines.py", line 1316, in <listcomp>
    for s, e, score in zip(starts, ends, scores)
KeyError: 13
```

## Expected behavior

Predictions that are pointing to tokens that are not part of the "context" (here: the last [SEP] token) should be filtered out from possible answers. 

## Environment info

     
- `transformers` version: 3.0.2
- Platform: Ubuntu 18.04
- Python version: 3.7.6
- PyTorch version (GPU?): 1.5.1, CPU
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


Comments:
Hi @tholor, 

Thanks for reporting the issue. 

We did have an issue where predictions were going out of bounds on QA pipeline and it has been fixed on master: 

```python
>>> nlp = pipeline("question-answering",model="distilbert-base-uncased-distilled-squad",
                                  tokenizer="distilbert-base-uncased",
                                  device=-1)

>>> nlp(question="test finding", context="My name is Carla and I live in Berlin")
>>> {'score': 0.41493675112724304, 'start': 11, 'end': 16, 'answer': 'Carla'}
```

If you are able to checkout from master branch I would be happy to hear back from you to make sure it's working as expected on your side as well.

Let us know 😃 
Morgan

Hi @mfuntowicz , 
Works like a charm now. Thanks for the fix!
[wip] T5 tokenizer should add special tokens
Author: sshleifer
Pull request: yes


Comments:
Thanks for taking care of it @sshleifer !

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5832?src=pr&el=h1) Report
> Merging [#5832](https://codecov.io/gh/huggingface/transformers/pull/5832?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/615be03f9d961c0c9722fe10e7830e011066772e&el=desc) will **decrease** coverage by `0.18%`.
> The diff coverage is `84.61%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5832/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5832?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5832      +/-   ##
==========================================
- Coverage   78.66%   78.48%   -0.19%     
==========================================
  Files         146      146              
  Lines       26200    26213      +13     
==========================================
- Hits        20611    20574      -37     
- Misses       5589     5639      +50     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5832?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/tokenization\_t5.py](https://codecov.io/gh/huggingface/transformers/pull/5832/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdDUucHk=) | `94.04% <84.61%> (-1.73%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_electra.py](https://codecov.io/gh/huggingface/transformers/pull/5832/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9lbGVjdHJhLnB5) | `26.02% <0.00%> (-69.52%)` | :arrow_down: |
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5832/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `86.21% <0.00%> (-0.51%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_distilbert.py](https://codecov.io/gh/huggingface/transformers/pull/5832/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9kaXN0aWxiZXJ0LnB5) | `98.79% <0.00%> (+33.89%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5832?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5832?src=pr&el=footer). Last update [615be03...04f18d4](https://codecov.io/gh/huggingface/transformers/pull/5832?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
[seq2seq] distillation.py accepts trainer arguments
Author: sshleifer
Pull request: yes

cc @nateraw 

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5865?src=pr&el=h1) Report
> Merging [#5865](https://codecov.io/gh/huggingface/transformers/pull/5865?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/529850ae7bca0ff388778c3c0d66240834cf56c3&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5865/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5865?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #5865   +/-   ##
=======================================
  Coverage   78.48%   78.48%           
=======================================
  Files         146      146           
  Lines       26200    26200           
=======================================
  Hits        20563    20563           
  Misses       5637     5637           
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5865?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5865?src=pr&el=footer). Last update [529850a...259bc29](https://codecov.io/gh/huggingface/transformers/pull/5865?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Xlnet outputs
Author: TevenLeScao
Pull request: yes

Reopening #5770 since push failed.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5881?src=pr&el=h1) Report
> Merging [#5881](https://codecov.io/gh/huggingface/transformers/pull/5881?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/3653d01f2af0389207f2239875a8ceae41bf0598&el=desc) will **increase** coverage by `0.54%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5881/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5881?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5881      +/-   ##
==========================================
+ Coverage   77.26%   77.81%   +0.54%     
==========================================
  Files         146      146              
  Lines       25948    25958      +10     
==========================================
+ Hits        20048    20198     +150     
+ Misses       5900     5760     -140     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5881?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/configuration\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5881/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX3V0aWxzLnB5) | `96.52% <ø> (ø)` | |
| [src/transformers/configuration\_xlnet.py](https://codecov.io/gh/huggingface/transformers/pull/5881/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX3hsbmV0LnB5) | `94.33% <ø> (ø)` | |
| [src/transformers/modeling\_xlnet.py](https://codecov.io/gh/huggingface/transformers/pull/5881/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ194bG5ldC5weQ==) | `81.75% <100.00%> (+0.26%)` | :arrow_up: |
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5881/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `86.21% <0.00%> (-0.26%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_distilbert.py](https://codecov.io/gh/huggingface/transformers/pull/5881/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9kaXN0aWxiZXJ0LnB5) | `98.79% <0.00%> (+33.89%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5881?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5881?src=pr&el=footer). Last update [3653d01...cfd7c66](https://codecov.io/gh/huggingface/transformers/pull/5881?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
How to use DistilBertTokenizer in C++
Author: alxmamaev
Pull request: no
Labels: name: wontfix

I saw that fast tokenizers was realized as Rust library, but I not fund bindings for C++, and also I cannot use a pretrained distilled tokenizer in fast mode.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Fine-tuning for paraphrasing tasks
Author: anmoljagetia
Pull request: no
Labels: name: Discussion

name: wontfix

# ❓ Questions & Help
I asked on SO and was downvoted since it is considered "off-site" and is against their Terms of Service. My question is somewhat simple. How do I fine-tune a GPT-2 model for the task of paraphrasing like the paper: https://www.aclweb.org/anthology/D19-5623.pdf

A link to my SO question : https://stackoverflow.com/questions/61115488/how-to-fine-tune-gpt-2-for-paraphrasing?noredirect=1#comment108120354_61115488

## Details
My question is somewhat simple. How do I fine-tune a GPT-2 model for the task of paraphrasing like the paper: https://www.aclweb.org/anthology/D19-5623.pdf

Is there a way to achieve this with huggingface-transformers ?


Comments:
This might help: https://huggingface.co/transformers/usage.html#sequence-classification

^^ These are inference examples. Do you know how can I _retrain_ ?

I would stress that this topic is quite interesting and useful. A good generative model for paraphrasing may help with text classification with small datasets. Backtranslation (for example) has shown as an effective way to augment the training data and boost performance of a classifier. However, echoing the @anmoljagetia, fine-tuning on the target domain may also bee important. 



@anmoljagetia did you find any method to retrain the model to generate paraphrase sentence?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Seq2Seq: Option to not store whole dataset in memory
Author: sshleifer
Pull request: no
Labels: name: Help wanted

https://github.com/huggingface/transformers/blob/ef0e9d806c51059b07b98cb0279a20d3ba3cbc1d/examples/seq2seq/utils.py#L93

Comments:
Do you still need help? I can help out and contribute here. 

Yes that would be super helpful! The goal is to avoid using so much CPU memory here:
https://github.com/huggingface/transformers/blob/353b8f1e7a7361c0afd9e391381bc226b4a5ca8f/examples/seq2seq/utils.py#L101

by only reading a few batches from disk at a time instead of all at once. Will probably require saving in a different format.
Maybe something like this:
https://github.com/pytorch/fairseq/blob/f0a61a2774aff2efbc1adb0b5daee346a8401605/fairseq/data/data_utils.py#L55

Let me know if you need more info!

Great! 
My idea is to lazily read & encode just the required line numbers from the file when `__getitem__` is called with an index. For this we could create a map of {example number: line numbers} to read. Let me know what you think.

Sounds reasonable. What does fairseq do?

Your approach sounds good, feel free to get started.

Another approach would be not pad inputs when they are getting cached and then make a batch at load time. 

I think fairseq had the data in multiple files instead of one big one. Sounds good - I am working on it. Will share when I have a tested version. 

Couple of questions : I plan to get rid of the self.source variable; 
1. can I get rid of this property?https://github.com/huggingface/transformers/blob/353b8f1e7a7361c0afd9e391381bc226b4a5ca8f/examples/seq2seq/utils.py#L146-L148

2. Any ideas on how to use the sampler without the full dataset? In general shuffling and sampling may be limited with lazy datasets: although you should be able to use a random sampler in your loader.
https://github.com/huggingface/transformers/blob/353b8f1e7a7361c0afd9e391381bc226b4a5ca8f/examples/seq2seq/utils.py#L155

@sshleifer I have a working solution that works for the rest of the training loop and passes the tests. See [here](https://github.com/huggingface/transformers/compare/master...Pradhy729:lazy_loading_seq2seq)
Just need input on my points above. Let me know.

1) You can get rid of `src_lens` and `tgt_lens`, they are unused afaict

2) I would suggest trying to store the len of each example, (tokenized or untokenized, but not padded), and passing that to `SortishSampler` instead of `self.source`, and then changing
`def key(self, i): len(self.data[i])` -> `def key(self, i): self.data[i]`.

https://github.com/huggingface/transformers/blob/7e86d070c0bbed949b5c922f914f0fec44af72d4/examples/seq2seq/utils.py#L203.



Like https://github.com/huggingface/transformers/pull/5818

Got it. Good idea!
issue with loading pretrained model - xlnet
Author: SantalScript
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->
I ran the following code: 
from transformers import XLNetTokenizer, XLNetForSequenceClassification
import torch
tokenizer = XLNetTokenizer.from_pretrained('xlnet-base-cased')
model = XLNetForSequenceClassification.from_pretrained('xlnet-base-cased')

## Details
<!-- Description of your issue -->
and I have the following error: 

> OSError: Can't load config for 'xlnet-base-cased'. Make sure that:

> 'xlnet-base-cased' is a correct model identifier listed on 'https://huggingface.co/models'

> or 'xlnet-base-cased' is the correct path to a directory containing a config.json file


<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
The error didn't appear when I ran the same piece of code two days ago so I think it might be caused by recent update. 

Comments:
Deal with multiple choice in common tests
Author: sgugger
Pull request: yes

It's a bit heavy but I didn't find another way to reshape the inputs when needed for the multiple choice model. With this, and skipping the input_embeds test when the model is a multiple choice one (current implementation requires `input_ids`), I manage to have the common tests passing for `BertForMultipleChoice`.

Let me know if you have other ideas!

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4886?src=pr&el=h1) Report
> Merging [#4886](https://codecov.io/gh/huggingface/transformers/pull/4886?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/02e5f79662d72cccdca81a47e3001a5f6d36e5b1&el=desc) will **increase** coverage by `0.09%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4886/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4886?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4886      +/-   ##
==========================================
+ Coverage   76.46%   76.56%   +0.09%     
==========================================
  Files         128      128              
  Lines       21502    21502              
==========================================
+ Hits        16442    16463      +21     
+ Misses       5060     5039      -21     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4886?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/4886/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19yb2JlcnRhLnB5) | `94.73% <0.00%> (+10.04%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4886?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4886?src=pr&el=footer). Last update [02e5f79...b78ed3a](https://codecov.io/gh/huggingface/transformers/pull/4886?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).