noamaanMulla-03/transformers-issues huggingface dataset:
Errors when using fp16 with traced models
Author: chessgecko
Pull request: no
Labels: name: wontfix

## 🐛 Bug

When I run

```
roberta_model = RobertaForMaskedLM.from_pretrained("roberta-base", torchscript=True)
roberta_model.cuda()
roberta_model.half()
traced_model = torch.jit.trace(roberta_model, (r_input_ids))
```

I get the following error

`
Expected object of scalar type Float but got scalar type Half for argument #2 'mat2'
`

When I attempt to load a normally traced model

```
loaded_model = torch.jit.load("traced_roberta_cuda.pt")
loaded_model.cuda()
loaded_model.half()
loaded_model(r_input_ids )
```

I get

`RuntimeError: expected device cuda:0 and dtype Float but got device cuda:0 and dtype Half
`

Is there a way to use fp16 with traced models? It happened with BertForSequenceClassification, RobertaForSequenceClassification and RobertaForMaskedLM.

## Environment
* Models tested on: Bert and Roberta:
* Language: English
* OS: Ubuntu 18.04
* Python version: 3.6.9
* PyTorch version: 1.2.0
* PyTorch Transformers version (or branch): 2.0.0


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Has this ever been solved? I have the same issue

I think maybe the reason is `.half()` only change(cast) the data, but it is not traceable. And there is no fp32 -> fp16 in python code, the function will expect fp32 input instead of fp16.

Just confirming I am still using this in production and never found a solution. If there is an easy solution here I'd happily pay a small bounty for the information.

Just stumbled across this trying to look for anything talking about using fp16 precision with torchscript. Converting the model and inputs to half seems to work. I get a lot higher warnings about loss of precision with torchscript when I have `use_fp16=True`.  Not sure if I'm being paranoid with the `torch.no_grad()` statement, I don't know if it'll do that internally within `torch.jit.trace` but I couldn't see anything about it.

```python
model = model.cuda()
model.eval()

with torch.no_grad():
    inputs = torch.randn(input_shape, device='cuda')
    
    if use_fp16:
        model = model.half()
        inputs = inputs.half()

    traced_model = torch.jit.trace(model, inputs)
```

It's kind of strange. When I don't check the trace during tracing and call inference without ``torch.no_grad()`` it does actually work (but consumes way too much memory of course because of gradient computations).

```
model.half()
model = torch.jit.trace(model, (dummy_input, dummy_input, dummy_input), check_trace=False)

outputs = model(inputs)
```

Actually I also have another issue with TorchScript, because I cannot feed the inputs as dict during tracing. In the case of BERT it then somehow uses ``input_ids, attention_mask, inputs`` as input names instead of ``input_ids, attention_mask, token_type_ids``.

For someone who encounters the same problem, this issue is fixed in torch 1.5 and present back in torch 1.6.
getting 'ValueError-TextInputSequence must be str' in 'train_dataset = train_dataset.map(convert_to_features)'
Author: hemantwani
Pull request: no

Hello friends,
I want to fine tune longformer QnA for different data. I found this code very relevant to my requirement, so was trying to understand the same thoroughly. but getting an error. 
i am trying to run the same code that Suraj has on github.

**

> 'ValueError-TextInputSequence must be str' in 'train_dataset = train_dataset.map(convert_to_features)'.

**

`train_dataset  = nlp.load_dataset('squad', split=nlp.Split.TRAIN)
valid_dataset = nlp.load_dataset('squad', split=nlp.Split.VALIDATION)


train_dataset = train_dataset.map(convert_to_features)
valid_dataset = valid_dataset.map(convert_to_features, load_from_cache_file=False)


#patil_suraj

Comments:
@patil-suraj 
Can you help me with this?
Index out of range in Bart-large-xsum
Author: tejareddy8888
Pull request: no

Questions & Help

Hello to everyone!!

I am facing a problem summarizing long articles. I mean very long text with larger vocab size than it is pre-trained already i guess.
I see that many of the models have a limitation of maximum input and trying to execute results in error of index out of range.
I am particularly using "BART-large-xsum". Please suggest what is the correct way of using these models with long documents shall I finetuning to increase the vocabsize or do anything else.

A code snippet with an example of how to handle long documents with the "BART-large-xsum" would be perfect to start with!

Thanks in advance,
Teja


Comments:
Pinging @sshleifer, the summarization master

See https://discuss.huggingface.co/t/summarization-on-long-documents/920/2, and feel free to reply there!
I don't have a code snippet but feel free to contribute one to that discussion!
SPM Tokenizer confusion with fairseq Roberta
Author: hichiaty
Pull request: no

Hi,

I have pre-trained a custom Roberta Model from scratch with a unigram sentencepiece model (also trained from scratch). I have converted the model from fairseq to huggingface with this [script](https://github.com/huggingface/transformers/blob/master/src/transformers/convert_roberta_original_pytorch_checkpoint_to_pytorch.py) successfully.

I have tried loading the model in huggingface, which was successful, but the issue lies with the tokenizer, I tried using the Roberta tokenizer but it screamed at me because it was looking for merges and vocab. I then loaded my spm model with AlbertTokenizer, but when I try to test it out with a simple fill_mask, the answer tokens are incorrect. How do I correctly use my SPM model with Roberta? I also have dict.txt from fairseq.

Any help would be appreciated!

Comments:
Hi @hichiaty ,

in that case I would try the CamemBERT Tokenizer or the one from XLM-RoBERTa (last one contains some hacky workaround to align fairseq vocab with SPM...) :)

hey @stefan-it  thanks for the advice! I am still slightly confused though, I loaded my spm model with CamemBERT but for some reason it still doesn't match the tokens from fairseq's roberta.encode.

Using Fairseq's encode I get:

```
roberta.encode('HELLO')
>tensor([0, 7, 4, 6, 2]) 
```
With CamemBERT I get:

```
tokenizer('HELLO')
>{'input_ids': [5, 45, 36, 3863, 3595, 3595, 19, 5], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1]}
```
I'm just trying to figure out how fairseq works at this point because I don't even pass the spm model to it but a dict.txt file

Solved, created a custom tokenizer (based on camembert) that uses spm to encode as pieces, then fairseq's dict.txt to get ids.
non-interactive transformers-cli upload?
Author: stas00
Pull request: no

# 🚀 Feature request

Currently, `transformers-cli upload` works only interactively due to its prompt:
`Proceed? [Y/n]`

After running the updated model conversion, I would like to be able to do:
```
cd data
transformers-cli upload fsmt-wmt19-ru-en
transformers-cli upload fsmt-wmt19-en-ru
transformers-cli upload fsmt-wmt19-de-en
transformers-cli upload fsmt-wmt19-en-de
cd -
```
But this won't work:

Would it be possible to add a `-y` override?

Alternatively, would it be possible to give it all dirs to upload in one command?
```
transformers-cli upload fsmt-wmt19-ru-en fsmt-wmt19-en-ru fsmt-wmt19-de-en fsmt-wmt19-en-de
```

## Motivation

I have been re-uploading 4 x 1.1GB models on a relatively slow connection, and I have to be around to hit Y for each one of them, which is very counter-productive, as I have to go back and re-check whether each upload has been completed. I can probably code some shell expect script to feed it automatically, but this defeats the purpose.

Thank you!


Comments:
FWIW, I found a workaround (god bless Stackoverflow):

```
cd data
yes Y | transformers-cli upload fsmt-wmt19-ru-en
yes Y | transformers-cli upload fsmt-wmt19-en-ru
yes Y | transformers-cli upload fsmt-wmt19-de-en
yes Y | transformers-cli upload fsmt-wmt19-en-de
cd -
```

Ah, nice find :)

I think a `-y` flag would be reasonable if you want to open a PR

Will do. Thank you.

Done: https://github.com/huggingface/transformers/pull/7035
There is an error in the run_tf_ner.py script with the get_labels fun…
Author: RodSernaPerez
Pull request: yes

There is an error in the run_tf_ner.py scripts with the get_labels function (does not exist). It does not work with TPU. Both problems are fixed.

Comments:
Thanks for the PR.

The issue with the `get_labels` has been fixed in a previous PR. Can you rebase on master and remove the `get_labels` part? We will keep the TPU fix :)

Closing as already implemented. Thanks a lot for your contribution @RodSernaPerez, looking forward to the next one!
Finetune other models for sentence-classification
Author: cmdllx
Pull request: no

# ❓ Questions & Help

I want to use run_glue.py to finetune pre-trained models for classification,but I find that the script can only be used for  BERT, XLM, XLNet and RoBERTa.I want to finetune other models like longformer ,what should I do?



Comments:
Hi, there is a `LongformerForSequenceClassification` so you should be able to use `run_glue.py` with that model.

I've removed the misleading dosctring in https://github.com/huggingface/transformers/commit/1650130b0fd71eb80380c47c8ffed89d49ff3481.

thank you.I can finetune all models for downstream tasks

sorry to interrupt you, I have a question for new version of transformers.
I once used the old version, when I run the python "run_glue.py", I found the time for loading different models to GPU is defferent(the code is model.to_device()&nbsp; &nbsp;),the more layer the model has, the more time it takes to load the model to GPU
However, when I use the new version,when I run the python "run_glue.py", I found the time for loading different models to GPU is nearly the same . (I think the new script load the model to GPU when init the Trainer,&nbsp;)
can you explain the reason&nbsp;&nbsp;


cczhou
chenmodechenchen@qq.com



&nbsp;




------------------&nbsp;原始邮件&nbsp;------------------
发件人:                                                                                                                        "huggingface/transformers"                                                                                    <notifications@github.com&gt;;
发送时间:&nbsp;2020年9月7日(星期一) 晚上8:17
收件人:&nbsp;"huggingface/transformers"<transformers@noreply.github.com&gt;;
抄送:&nbsp;"沉默"<1658802578@qq.com&gt;;"Author"<author@noreply.github.com&gt;;
主题:&nbsp;Re: [huggingface/transformers] Finetune other models for sentence-classification (#6937)





 
Closed #6937.
 
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
fix to ensure that returned tensors after the tokenization is Long
Author: GeetDsa
Pull request: yes

<!-- This line specifies which issue to close after the pull request is merged. -->
Fixes #7026 


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/7039?src=pr&el=h1) Report
> Merging [#7039](https://codecov.io/gh/huggingface/transformers/pull/7039?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/762cba3bdaf70104dc17cc7ff0f8ce13ba23d558?el=desc) will **not change** coverage.
> The diff coverage is `0.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/7039/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/7039?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #7039   +/-   ##
=======================================
  Coverage   79.13%   79.13%           
=======================================
  Files         164      164           
  Lines       31143    31143           
=======================================
  Hits        24646    24646           
  Misses       6497     6497           
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/7039?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/data/data\_collator.py](https://codecov.io/gh/huggingface/transformers/pull/7039/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9kYXRhL2RhdGFfY29sbGF0b3IucHk=) | `93.18% <0.00%> (+0.35%)` | :arrow_up: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/7039/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `82.41% <0.00%> (-0.26%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/7039?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/7039?src=pr&el=footer). Last update [762cba3...db8ea89](https://codecov.io/gh/huggingface/transformers/pull/7039?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Wouldn't it be better to do `torch.tensor(e, dtype=torch.long)` when creating the tensor, instead of casting the batch once the tensors are created?

@LysandreJik, made the suggested changes. :+1: 

Thanks for the fix!
Usage of targets argument in fill-mask pipeline (Pipeline cannot handle mixed args and kwargs)
Author: mattjhill
Pull request: no

I'm trying to use the `targets`  keyword in the fill mask pipeline as described in #6239 but I'm getting a `ValueError: Pipeline cannot handle mixed args and kwargs`

Full example

    nlp = pipeline('fill-mask', topk=2)
    nlp("The acting was believable and the action was outstanding. The sentiment of this review is <mask>.", targets=[' positive', ' negative'])

    ---------------------------------------------------------------------------
    ValueError                                Traceback (most recent call last)
    <ipython-input-44-2f6d875aee9b> in <module>
    ----> 1 nlp("The acting was believable and the action was outstanding. The sentiment of this review is <mask>.",
        2     targets=[' positive', ' negative'])

    ~/miniconda3/envs/huggingface/lib/python3.8/site-packages/transformers/pipelines.py in __call__(self, *args, **kwargs)
    1054
    1055     def __call__(self, *args, **kwargs):
    -> 1056         inputs = self._parse_and_tokenize(*args, **kwargs)
    1057         outputs = self._forward(inputs, return_tensors=True)
    1058

    ~/miniconda3/envs/huggingface/lib/python3.8/site-packages/transformers/pipelines.py in _parse_and_tokenize(self, padding, add_special_tokens, *args, **kwargs)
        503         """
        504         # Parse arguments
    --> 505         inputs = self._args_parser(*args, **kwargs)
        506         inputs = self.tokenizer(
        507             inputs, add_special_tokens=add_special_tokens, return_tensors=self.framework, padding=padding,

    ~/miniconda3/envs/huggingface/lib/python3.8/site-packages/transformers/pipelines.py in __call__(self, *args, **kwargs)
        167     def __call__(self, *args, **kwargs):
        168         if len(kwargs) > 0 and len(args) > 0:
    --> 169             raise ValueError("Pipeline cannot handle mixed args and kwargs")
        170
        171         if len(kwargs) > 0:

    ValueError: Pipeline cannot handle mixed args and kwargs




Comments:
Looks like I was using v3.0.2 and this was introduced in v3.1.0
max_length does not seem to work
Author: MarijnQ
Pull request: no

So I may have understood the definition wrong, but in my understanding max_length defines the amount of characters you want your summary to be. i.e. max_length=50, the summary is not over 50 characters.

I have installed the transformers, work with the summarizer pipeline, which I assume is the bartsummarizer.
I try to summarize the standard text from the tutorial about the woman who married a lotta guys.
I fill in the following string:
`summarizer = pipeline('summarization')
summarizer(TEXT_TO_SUMMARIZE, min_length=50, max_length=100)`

The summary is this
` Liana Barrientos pleaded not guilty to two counts of "offering a false instrument for filing in the first degree" She has been married to 10 men, nine of them between 1999 and 2002 . At one time, she was married to eight men at once, prosecutors say .`
Which is 249 characters and has weird errors with the . being placed after a space.

It is probably a rookie question, but I can't seem to figure it out.

ps. I run it on Google Colab, if that makes a difference

Comments:
Hey @MarijnQ,

`max_length` defines the number of maximum output *tokens* (which is usually a bit less than number of words). You can get the number of tokens of a text by doing:

```python 
from transformers import T5Tokenizer

tokenizer = T5Tokenizer.from_pretrained("t5-base")
input_ids = tokenizer('Liana Barrientos pleaded not guilty to two counts of "offering a false instrument for filing in the first degree" She has been married to 10 men, nine of them between 1999 and 2002 . At one time, she was married to eight men at once, prosecutors say .')

print(f"Num tokens {len(input_ids.input_ids)}")  
```
id2lang in tokenization_xlm.py should be int, and removing hardcoding
Author: stas00
Pull request: no

In https://github.com/huggingface/transformers/blob/master/src/transformers/tokenization_xlm.py#L78 we have:
```
        "id2lang": {"0": "de", "1": "en"},
        "lang2id": {"de": 0, "en": 1},
```
and then:
```
        lang2id (:obj:`Dict[str, int]`, `optional`, defaults to :obj:`None`):
            Dictionary mapping languages string identifiers to their IDs.
        id2lang (:obj:`Dict[int, str`, `optional`, defaults to :obj:`None`):
```

So it should be:
```
        "id2lang": {0: "de", 1: "en"},
        "lang2id": {"de": 0, "en": 1},
```
All other entries need this change too.

The problem hasn't been detected until now since they were used to only count the number of languages it seems.

I need to pass src/tgt languages to the tokenizer I'm porting from fairseq, so I was looking at how to do that and `id2lang` seems to fit the purpose. But I actually need to look them up by `int` id, that's how I saw the problem.

But I'm also not sure why do we need to hardcode the reversal, when it can be done in 1 line of code? Which would also remove this assertion code:

```
        self.lang2id = lang2id
        self.id2lang = id2lang
        if lang2id is not None and id2lang is not None:
            assert len(lang2id) == len(id2lang)
```

Further we don't even need to hardcode the ids. Replace:
```
       "id2lang": {0: "de", 1: "en"},
```
with:
```
       "id2lang": ["de", "en"]
```
So all we need is one of the two entries, and now generate the 2 lookup dicts on the fly.

And since it's no longer `id2lang` semantically, probably renaming it to just `langs` would be more appropriate.

I think I will use this approach regardless of the outcome of this issue.

Thanks.

Comments:
Not sure if the suggested rewrite to remove all those numbers is desirable - perhaps it's important to see those numbers, so I left it alone and just fixed the keys of `id2lang` to be int.

https://github.com/huggingface/transformers/pull/7034
DistributedSortishSampler
Author: sshleifer
Pull request: no
Labels: name: Help wanted

name: Examples

In examples/seq2seq/finetune.py, 
`--sortish_sampler  --gpus 2` will raise an assertion error, but if you remove the assert, it will raise another error.   Ideally we should make a Seq2SeqDataset.get_distributed_sortish_sampler method and use it in the relevant case.



Comments:
No documentation for MMBT on official docs
Author: gchhablani
Pull request: no
Labels: name: wontfix

I tried finding MMBT on the official documentation: https://huggingface.co/transformers/ but I could not find any references to it, even though there is an implementation for it in the source code.


Comments:
I also cannot figure out how to use this model.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Better TPU Support in examples
Author: YuxianMeng
Pull request: no
Labels: name: wontfix

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

I tried to train BERT on TPU recently, and found your [examples](https://github.com/huggingface/transformers/blob/master/examples) has done work on this topic. However, some code looks experimental and not perfectly ready to be used. The combination of [xla and pytorch-lightening](https://github.com/huggingface/transformers/blob/master/examples/lightning_base.py) looks great, but seems not be used neither in any training scripts nor in documents. I'd like to know when will those codes be ready, and I'd be very glad to contribute some code.


Comments:
I agree with this request. TPU training pipeline is very fragile, lacking and needs more attention. Encouraging more use of TPU by providing easy examples would increase its usage resulting in a higher quality TPU Training system over time as more people contribute.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Question about the test results of my own testsets with a fine-tuned BERT
Author: Deep1994
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
<!-- Description of your issue -->
Hi, I have a fine-tuned BERT model and I want to use it to predict on my own testset which have 40k sentence pairs. I use the script like:

`export GLUE_DIR=./glue_data
export TASK_NAME=QQP

CUDA_VISIBLE_DEVICES=3 python ./examples/text-classification/run_glue.py \
    --model_name_or_path ./pretrained_weights/bert_base_uncased \
    --task_name $TASK_NAME \
    --do_predict \
    --data_dir $GLUE_DIR/$TASK_NAME \
    --max_seq_length 128 \
    --per_device_eval_batch_size=32   \
    --per_device_train_batch_size=32   \
    --learning_rate 2e-5 \
    --num_train_epochs 3.0 \
    --overwrite_output_dir  \
    --output_dir ./tmp/filter_nouns_pairs \`

It stands to reason that I should get a prediction file containing 40k rows, but in fact the output result is still only 10,000 rows. Is the result truncated during prediction? Thanks!
<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:

Comments:
I found the cause of the problem. If you want to predict on your own test set, you need to delete the previous "cached_test_BertTokenizer_128_qqp.lock" file and "cached_test_BertTokenizer_128_qqp" file.