noamaanMulla-03/transformers-issues huggingface dataset:
Create README.md
Author: jannesgg
Pull request: yes
Labels: name: model card


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3306?src=pr&el=h1) Report
> Merging [#3306](https://codecov.io/gh/huggingface/transformers/pull/3306?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/68ef0a111f8740f06ca4e5a00374ec4e2adb0a6d&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3306/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3306?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3306      +/-   ##
==========================================
- Coverage   77.48%   77.47%   -0.02%     
==========================================
  Files          99       99              
  Lines       16799    16799              
==========================================
- Hits        13017    13015       -2     
- Misses       3782     3784       +2     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3306?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3306/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `85.68% <0.00%> (-0.54%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3306/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `93.84% <0.00%> (+0.13%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3306?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3306?src=pr&el=footer). Last update [68ef0a1...6053b4e](https://codecov.io/gh/huggingface/transformers/pull/3306?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Thanks for contributing @jannesgg – if this is a Swedish LM, could you add the language tag to the top of the model card:

```
---
language: swedish
---
```

Thanks! [`Model page`](https://huggingface.co/jannesg/bertsson)
UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 13: character maps to <undefined>
Author: gilmartenorio
Pull request: no
Labels: name: wontfix

I am trying to export a query to a .csv file, but I am having some issues. Here is the code:

import pandas as pd
import cx_Oracle as cx_Oracle

print("Efetuando login...")
dsn_tns = cx_Oracle.makedsn(r'bdproddr-exad.pitagoras.apollo.br','1521', service_name='bdprodexa')
conn = cx_Oracle.connect(user=r'UserName',password = 'xxxx',dsn=dsn_tns)
print('Usuário logado.')

c = conn.cursor()

print("A extração esta sendo feita, por favor aguardar...")

try:
    query = ''' Here goes the SQL code '''
df2 = pd.read_sql(con = conn, sql = query)
finally:
    conn.close()

df2.head ()
print ('Exportando dados para arquivo CSV...')
df2.to_csv(r'Z:\1 - EQUIPE_GPA\BASES_AEDU_DM_FAMA\Extração_Diaria\ExtracaoBaseDiaria_DM_AEDU_Pais_Filhos.csv', encoding = 'utf-16')

When I try to run I receive the following error:
**Traceback (most recent call last):
  File "C:\Users\gilmar.melo\OneDrive - EDITORA E DISTRIBUIDORA EDUCACIONAL S A\Python\Consultas\ExtracaoBaseDiaria_DM_AEDU_Pais_Filhos.py", line 82, in <module>
    df2 = pd.read_sql(con = conn, sql = query)
  File "C:\Users\gilmar.melo\AppData\Local\Programs\Python\Python38-32\lib\site-packages\pandas\io\sql.py", line 404, in read_sql
    return pandas_sql.read_query(
  File "C:\Users\gilmar.melo\AppData\Local\Programs\Python\Python38-32\lib\site-packages\pandas\io\sql.py", line 1658, in read_query
    data = self._fetchall_as_list(cursor)
  File "C:\Users\gilmar.melo\AppData\Local\Programs\Python\Python38-32\lib\site-packages\pandas\io\sql.py", line 1671, in _fetchall_as_list
    result = cur.fetchall()
  File "C:\Users\gilmar.melo\AppData\Local\Programs\Python\Python38-32\lib\encodings\cp1252.py", line 15, in decode
    return codecs.charmap_decode(input,errors,decoding_table)
UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 13: character maps to <undefined>**

I tried a similar code for another query and worked, I only have problem with this particularly.


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Any example of how to do multi-class classification with TFBertSequenceClassification
Author: usmanmalik57
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

I am trying to create a multi-class text classification model using BertSequenceClassifier for Tensorflow 2.0. Any help with the implementation strategy would be appreciated. Also, are there any recommendations as to how to convert a simple CSV file containing text and labels into TF dataset format such as Glue?

Comments:
im asking for the same thing

I also need this. experts, please help/guide.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Add model card for Google AI's BERT Miniatures
Author: iuliaturc-google
Pull request: yes
Labels: name: model card

This model card is intended to be shared among all models under google/bert_uncased_*
(We'll need some support from HuggingFace to get this card cross-linked from all models)

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3301?src=pr&el=h1) Report
> Merging [#3301](https://codecov.io/gh/huggingface/transformers/pull/3301?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/47591763137f17021928e686ef171f25c240f076&el=desc) will **decrease** coverage by `0.35%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3301/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3301?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3301      +/-   ##
==========================================
- Coverage   77.68%   77.33%   -0.36%     
==========================================
  Files          99       99              
  Lines       16799    16799              
==========================================
- Hits        13051    12991      -60     
- Misses       3748     3808      +60     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3301?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/data/processors/squad.py](https://codecov.io/gh/huggingface/transformers/pull/3301/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9kYXRhL3Byb2Nlc3NvcnMvc3F1YWQucHk=) | `28.89% <0.00%> (-6.50%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3301/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `82.46% <0.00%> (-5.91%)` | :arrow_down: |
| [src/transformers/tokenization\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3301/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdXRpbHMucHk=) | `91.01% <0.00%> (-0.99%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3301?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3301?src=pr&el=footer). Last update [4759176...46b9c45](https://codecov.io/gh/huggingface/transformers/pull/3301?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


I just symlinked all 24 under Google's namespace to this one in 68ef0a111f8740f06ca4e5a00374ec4e2adb0a6d.

Thanks for uploading the models @iuliaturc-google and @srush!

Example model page: https://huggingface.co/google/bert_uncased_L-4_H-128_A-2

Thanks Sasha and Julien!
[generate] do_sample default back to False
Author: patrickvonplaten
Pull request: yes

This somewhat reverts the commit:
https://github.com/huggingface/transformers/commit/6c1b23554f8bb5b5e1f6c80969acab764c755678

and the decision taken in #2696

and sets the default sampling behavior of `generate()` to greedy - / beam search. 

Pros:
- `False` is the more natural default value 
- Prettier API (especially for encoder_decoder models which will mostly only use beam search generate()) 

Cons:
- Some people might aleady be used to the `do_sample=True` default value and this commit might break the logic of their code (but would be trivial to change for them) 

I'm somewhat indifferent whether this PR should be merged, but I think @thomwolf and @sshleifer  are in favor of it. 

@LysandreJik @thomwolf @sshleifer 

Comments:
Not 100% sure how this results in a "Prettier API," but agree this isn't a big deal to fix downstream. (my current code explicitly sets `do_sample=True` just in case something like this happened.)

If you are creating any demo generation notebooks/tooling like Write With Transformer, I recommend explicitly noting this behavior.
[WIP] Lightning glue example
Author: nateraw
Pull request: yes

This PR adds an example of using Pytorch Lightning to run the GLUE benchmark. Additionally, I altered the `transformer_base.py` to use auto models and moved it to the example directory so it could be copied in by any script that wishes to use it.

Preferably, the base transformer would have subclasses for the different types of tasks, but I just used a dictionary with a key passed on init instead. (i.e. NER uses `AutoModelForTokenClassification` and GLUE uses `AutoModelForSequenceClassification`).

Comments:
@srush Can you please take a look?

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3290?src=pr&el=h1) Report
> Merging [#3290](https://codecov.io/gh/huggingface/transformers/pull/3290?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/8320feec09309a94f673e1e7ce2a93da81eb3366&el=desc) will **increase** coverage by `0.18%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3290/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3290?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3290      +/-   ##
==========================================
+ Coverage   77.81%   77.99%   +0.18%     
==========================================
  Files          98       98              
  Lines       16666    16666              
==========================================
+ Hits        12969    12999      +30     
+ Misses       3697     3667      -30     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3290?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3290/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `93.84% <0.00%> (+0.27%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3290/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `87.47% <0.00%> (+5.00%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3290?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3290?src=pr&el=footer). Last update [8320fee...dd1b783](https://codecov.io/gh/huggingface/transformers/pull/3290?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Looks excellent. I will let @LysandreJik merge tomorrow, and confirm multi-gpu / TPU work. 

Want to try SQuAD next?

> Want to try SQuAD next?

Sure, I'll give it a go.
Create model card for CodeBERTaPy
Author: mrm8488
Pull request: yes


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3309?src=pr&el=h1) Report
> Merging [#3309](https://codecov.io/gh/huggingface/transformers/pull/3309?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/2187c49f5cde57306c3fd1eb67dbc68fab9c6403&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3309/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3309?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3309      +/-   ##
==========================================
+ Coverage   76.92%   76.93%   +0.01%     
==========================================
  Files         100      100              
  Lines       16953    16953              
==========================================
+ Hits        13041    13043       +2     
+ Misses       3912     3910       -2     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3309?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3309/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.09% <0.00%> (+0.26%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3309?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3309?src=pr&el=footer). Last update [2187c49...003d51b](https://codecov.io/gh/huggingface/transformers/pull/3309?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Update examples/ner/run_ner.py to use AutoModel
Author: lifefeel
Pull request: yes

This PR is updating `run_ner.py` to use AutoModel implementation. Refer to #3290 and is simpler than before.

Maybe @srush can review this.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3305?src=pr&el=h1) Report
> Merging [#3305](https://codecov.io/gh/huggingface/transformers/pull/3305?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/b2028cc26b61a9dad960274d427e261af7c9bdc8&el=desc) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3305/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3305?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3305      +/-   ##
==========================================
- Coverage   77.47%   77.46%   -0.01%     
==========================================
  Files          99       99              
  Lines       16799    16799              
==========================================
- Hits        13015    13014       -1     
- Misses       3784     3785       +1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3305?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3305/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `85.68% <0.00%> (-0.36%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3305/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `93.70% <0.00%> (+0.13%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3305?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3305?src=pr&el=footer). Last update [b2028cc...85e70d9](https://codecov.io/gh/huggingface/transformers/pull/3305?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
GPT2Tokenizer doesn't include BOS or EOS token
Author: moinnadeem
Pull request: no

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): GPT-2

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

Script:
```
from transformers import GPT2Tokenizer

tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
encoded_dict = tokenizer.encode_plus(text="Hello I am Moin", add_special_tokens=True, \
    max_length=512, truncation_strategy="longest_first", pad_to_max_length=False, \
    return_tensors=None, return_token_type_ids=True, return_attention_mask=True, \
    return_overflowing_tokens=False, return_special_tokens_mask=False)

print(tokenizer.bos_token_id)
print(encoded_dict['input_ids'])
```

You should see that the `input_ids` do not include the `bos_token_id`. Shouldn't `encode_plus` be doing this?

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
The 
 token would appear, since I included to `add_special_tokens`.

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version:
- Platform: Linux-4.15.0-54-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.2
- PyTorch version (GPU?): 1.3.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


Comments:
Accidental double post -- closing this in favour of #3311
Mismatch in the accuracy figures
Author: thak123
Pull request: no

# ❓ Questions & Help

Hi Just wanted to know if "bert-base-multilingual-cased" model and https://storage.googleapis.com/bert_models/2018_11_23/multi_cased_L-12_H-768_A-12.zip are same. 

I tried using the tokenizer from the multi_cased_L-12_H-768_A-12 and bert-base-multilingual-cased model which performed better than using the tokenizer and model from pretrained  "bert-base-multilingual-cased" .

I was trying to fine tune a sentiment analysis task and the performance of using tokenizers above provide different performances.

Can anybody shed light on this ?

Comments:
Non-Deterministic Behavior in BertTokenizer
Author: 4tywon
Pull request: no

## 🐛 Bug

<!-- Important information -->

Model I am using (Bert, XLNet....): BertTokenizer only

Language I am using the model on (English, Chinese....):

The problem arise when using:
* [ ] the official example scripts: (give details)
* [X] my own modified scripts: Jupyter Notebook 

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: (give details) tokenize a string

## To Reproduce
Jupyter Notebook with one cell, with a cloned version of the transformers repo.

```
import sys
sys.path.insert(0, 'transformers')

from transformers import BertTokenizer

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased', sep_token='[SEP]', do_lower_case=True)

tokenizer.tokenize("[PAD] [SEP] [SEP] [PAD]")
```


with outputs (varying between kernel restarts and runs):
`['[PAD]', '[', 'sep', ']', '[', 'sep', ']', '[PAD]']`
`['[PAD]', '[SEP]', '[SEP]','[PAD]']`
`['[PAD]', '[SEP]', '[SEP]', '[', 'pad', ']']`
## Expected behavior
Expected the output to be 
`['[PAD]', '[SEP]', '[SEP]','[PAD]']`
and have deterministic behavior

<!-- A clear and concise description of what you expected to happen. -->

## Environment

Platform Linux-3.10.0-1062.el7.x86_64-x86_64-with-redhat-7.7-Verona
Python 3.7.4 (default, Aug 13 2019, 20:35:49) 
[GCC 7.3.0]
PyTorch 1.3.1
Tensorflow 1.15.0

* PyTorch Transformers version (or branch): on master branch of repo
* Using GPU ? Yes 

## Additional context

This happens sometimes when the notebook kernel is restarted and the cell is re-run. I haven't observed this happening when running a python script.

<!-- Add any other context about the problem here. -->


Comments:
As an update I've been able to reproduce this problem using a python script

I cannot reproduce this on Windows, PT1.4, latest transformers master.

I think I was using an old version of transformers :( 

This seems to have been fixed in v2.2.2 - After upgrading to latest I haven't observed this anymore so I'll close this issue.

Thanks!
Probably a bug in XLMRobertaTokenizer
Author: zjujh1995
Pull request: no
Labels: name: Core: Tokenization

(Everything goes perfect when I did experiment with MultilingualBert, but seems only the base-model is released.)

When using XLM-R, the according tokenizer (XLMRobertaTokenizer) converts \<unk\> and every OOV token into id = 1. However, 1 should be the number of \<pad\>. (And the tokenizer can convert 1 to \<pad\>, 3 to \<unk\>).

Comments:
Hi, if I am not wrong the pad should be 2 ?
at least the parameters tokenizer.pad_token_id for XLM-R 

EDIT: 2 is eos sorry.

Indeed, this is a bug that will be fixed when #3198 is merged. Thanks for letting us know.
XLM-Roberta mask filling error
Author: sultanovazamat
Pull request: no

# XLM-Roberta mask token filling error

Hi! I am trying to use XLM-Roberta for Masked LM task, but the error occurs when the model fills masked token in a test sentence.

**The code is:**

```
config.model_name = 'xlm-roberta-base'

tokenizer: tr.XLMRobertaTokenizer = tr.XLMRobertaTokenizer.from_pretrained(config.model_name)
model: tr.XLMRobertaForMaskedLM = tr.XLMRobertaForMaskedLM.from_pretrained(config.model_name)

input_ids = tokenizer.encode_plus("I want to <mask> New York!", 
                                               max_length=config.max_length)['input_ids']
x = np.full((config.max_length), fill_value=tokenizer.pad_token_id)
attn = np.zeros_like(x)
for i, tok in enumerate(input_ids):
    x[i] = tok
    attn[i] = 1
x = torch.tensor(x).unsqueeze(0).to(device)
attn = torch.tensor(attn).unsqueeze(0).to(device)

outputs = model(x, attention_mask=attn, masked_lm_labels=x)
```
**The error is**

```
RuntimeError: cublas runtime error : library not initialized at ../aten/src/THC/THCGeneral.cpp:216
```

When I try Albert for similar task everything works fine, but the Roberta family doesn't.
Could you please help with this issue?

Comments:
Found solution in [#2509](https://github.com/huggingface/transformers/pull/2509).

Hi, indeed this is an error. This will be fixed once #3198 is merged.