noamaanMulla-03/transformers-issues huggingface dataset:
Add camembert integration tests
Author: patrickvonplaten
Pull request: yes

Add integration tests for camembert comparing results to original fairseq code. 

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3375?src=pr&el=h1) Report
> Merging [#3375](https://codecov.io/gh/huggingface/transformers/pull/3375?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/8becb732931bbab5dd75cca5f5e7c75b2516d10b&el=desc) will **increase** coverage by `0.06%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3375/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3375?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3375      +/-   ##
==========================================
+ Coverage   77.64%   77.71%   +0.06%     
==========================================
  Files         100      100              
  Lines       16979    16979              
==========================================
+ Hits        13184    13195      +11     
+ Misses       3795     3784      -11     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3375?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3375/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.50% <0.00%> (+0.13%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3375/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `88.52% <0.00%> (+1.73%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3375?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3375?src=pr&el=footer). Last update [8becb73...74e09c3](https://codecov.io/gh/huggingface/transformers/pull/3375?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Model cards for CS224n SQuAD2.0 models
Author: elgeish
Pull request: yes
Labels: name: model card

For the following models:
* elgeish/cs224n-squad2.0-albert-base-v2
* elgeish/cs224n-squad2.0-albert-large-v2
* elgeish/cs224n-squad2.0-albert-xxlarge-v1
* elgeish/cs224n-squad2.0-distilbert-base-uncased
* elgeish/cs224n-squad2.0-roberta-base


Comments:
[model_cards] 🇹🇷 Add new BERTurk models
Author: stefan-it
Pull request: yes
Labels: name: model card

Hi,

this PR adds three new BERT models for Turkish:

* `dbmdz/bert-base-turkish-uncased` - uncased model with a vocab size of 32k
* `dbmdz/bert-base-turkish-128k-cased` - cased model with a vocab size of 128k
* `dbmdz/bert-base-turkish-128k-uncased` - uncased model with a vocab size of 128k

Models (incl. `tokenizer_config.json`) are already uploaded to the model hub :)

Results are coming soon in the [BERTurk repository](https://github.com/stefan-it/turkish-bert)! 

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3408?src=pr&el=h1) Report
> Merging [#3408](https://codecov.io/gh/huggingface/transformers/pull/3408?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/e392ba6938f50655a195ea7ec8a260b1e9fc6058&el=desc) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3408/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3408?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3408      +/-   ##
==========================================
- Coverage   77.56%   77.55%   -0.01%     
==========================================
  Files         100      100              
  Lines       16970    16970              
==========================================
- Hits        13162    13161       -1     
- Misses       3808     3809       +1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3408?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3408/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.58% <0.00%> (-0.14%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3408?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3408?src=pr&el=footer). Last update [e392ba6...756792d](https://codecov.io/gh/huggingface/transformers/pull/3408?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Add custom rules for sampling from GPT-2 Generator
Author: simonefrancia
Pull request: no

Hi @patrickvonplaten, 
I've read your [blogpost](https://huggingface.co/blog/how-to-generate) and it's really interesting. Thanks! I have a question for that. 
We have recently trained a GPT-2 generator with HF on general text and it works well.

But to have better results on my custom task, I would like to add a custom function of sampling that is able to maximize a certain behaviour of the text I am generating.
I would like to add a custom sampling applied to top-k words that are choosen by GPT-2 in order to maximize , for example, the fact that in the text I am generating there must be the max number of vocals. 
Could you help me to think about this solution?
Thanks

Comments:
Hey @simonefrancia happy that you liked the blog post :-) 

This sounds like quite a special sampling function, so the best you can do would be to fork/clone the repo and add this functionality yourself. If the sampling function is too special we probably will not include it into the master branch. 

But feel free to open a PR if you think it adds value and is quite general.
Bug in the command line tool: os.DirEntry not supported in Python 3.5
Author: netw0rkf10w
Pull request: no
Labels: name: wontfix

name: Core: CLI

Hi,

[This line](https://github.com/huggingface/transformers/blob/1a8e87be4e2a1b551175bd6f0f749f3d2289010f/src/transformers/commands/user.py#L162) will cause a syntax error in Python 3.5 as `os.DirEntry` does not exist.

You should either update the code for backward compatibility or update the README replacing 3.5+ by 3.6 (I believe the former would be preferred).

Best regards.


Comments:
Though the core of the library is Python 3.5+, the CLI is actually Python3.6+ as for instance the serving subcommand uses FastAPI which is Py36+ only (cc @mfuntowicz)

Do we have a way to make that clear in the doc @LysandreJik?

Personal opinion: supporting 3.6+ only seems realistic and may make maintenance easier. [AllenNLP](https://github.com/allenai/allennlp) also requires 3.6.1+. Of course I don't know how large the 3.5 user-base is for `transformers` so it might be worth maintaining. That being said, if there are plans to stop support for 3.5, it might be a good idea to announce this in a release ("last supported release for 3.5"). 

@BramVanroy According to the PyPI stats at https://pypistats.org/packages/transformers (look for the `Daily Download Proportions of transformers package - Python Minor` graph) around 1% of _pip installs_ are on Python 3.5.

> @BramVanroy According to the PyPI stats at https://pypistats.org/packages/transformers (look for the `Daily Download Proportions of transformers package - Python Minor` graph) around 1% of _pip installs_ are on Python 3.5.

Ah, I didn't know this website - thanks! I'm not sure if 1% is worth the effort, then again I don't know how much additional effort (and resources for CI) are needed to maintain for 3.5 anyway. (But 3.6 has f-strings and ordered dicts (officially in 3.7), PathLike, better `typing`,  sooo... :D)

PS: I wonder what happened on December 24 or thereabouts, with the spike in 3.5 installations. 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Closing this as the lib is now officially Python 3.6+
removing torch.cuda.empty_cache() from TF function
Author: keskarnitish
Pull request: yes

torch.cuda.empty_cache() was being called from a TF function (even when torch is unavailable)
not sure any replacement is needed if TF OOMs

simply running the benchmarks on a GPU with lower HBM will reproduce this error

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3267?src=pr&el=h1) Report
> Merging [#3267](https://codecov.io/gh/huggingface/transformers/pull/3267?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/cc4c37952a961f2d13e83f3d5ba6dab811d0bbfd&el=desc) will **not change** coverage by `%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3267/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3267?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #3267   +/-   ##
=======================================
  Coverage   77.82%   77.82%           
=======================================
  Files          98       98           
  Lines       16666    16666           
=======================================
  Hits        12970    12970           
  Misses       3696     3696           
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3267?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3267?src=pr&el=footer). Last update [cc4c379...4bb2bc3](https://codecov.io/gh/huggingface/transformers/pull/3267?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Indeed, thanks!
Added BioBERT-NLI model card
Author: gsarti
Pull request: yes
Labels: name: model card


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3421?src=pr&el=h1) Report
> Merging [#3421](https://codecov.io/gh/huggingface/transformers/pull/3421?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/d0c36a7b7270f114322c191866d29abea383e5da&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3421/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3421?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3421      +/-   ##
==========================================
+ Coverage   77.55%   77.56%   +0.01%     
==========================================
  Files         100      100              
  Lines       16970    16970              
==========================================
+ Hits        13161    13163       +2     
+ Misses       3809     3807       -2     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3421?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3421/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.85% <0.00%> (+0.27%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3421?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3421?src=pr&el=footer). Last update [d0c36a7...e17e0d2](https://codecov.io/gh/huggingface/transformers/pull/3421?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
when I install transformers, it appeared this error
Author: xiongma
Pull request: no

ERROR: Could not build wheels for tokenizers which use PEP 517 and cannot be installed directly

Comments:
You should add more information about your environment like OS, Python Version etc?
I just installed it on Windows 10, WSL etc Python 3.6.9 :: Anaconda, Inc.; It worked..

I installed it on Mac, Python 3.6.10 Anaconda
how to use TFBertModel to load a Bert, which the path is from own computer.
Author: xiongma
Pull request: no

my model's path is on my computer, so, I want to load it, but when I use TFBertModel to load it, it appeared this error.
`model = TFBertModel.from_pretrained('/Users/maxiong/Workpace/Code/transformers/pre_model',config=config)`

error:
![image](https://user-images.githubusercontent.com/30991932/76696213-dcf63400-66c3-11ea-8345-2d7780470709.png)


this is my model files
![image](https://user-images.githubusercontent.com/30991932/76696214-e384ab80-66c3-11ea-9b97-6a0a685f67d1.png)


Comments:
and how to use AutoTokenizer to load a local vocab file, instead of download a vocab file from server
a lot of examples in doc can't run successful
Author: xiongma
Pull request: no

I used example in doc, but a lot of example can't run successful, what's wrong?

Comments:
@patrickvonplaten my OS is mac OS 10.14.6, Python 3.6.10, tensorflow 2.0, transformers version is the resoruce code in github

Hi @policeme, which example did you use? 

![image](https://user-images.githubusercontent.com/30991932/76741738-050c9280-67ab-11ea-98a2-b3612b6695a9.png)
like this

@patrickvonplaten 

your example like this
![image](https://user-images.githubusercontent.com/30991932/76741937-5288ff80-67ab-11ea-8405-b30c191869fb.png)


Please don't paste screenshots of your code on issues. Copy and Paste the code in code format. We can't copy the code this way. Typing your code from a screenshot is very time-consuming. 

Considering the problem you have. How did you train the model that was saved in `...ckpt.index`? Did you use this library? From your error message, it seems like your Bert TF model saved in `.ckpt.index` does not have the correct form. 

The example you mention should be used if you trained your model with this library.

sorry about it, this is my code
`config = AutoConfig.from_pretrained(r'/Users/maxiong/Workpace/Code/transformers/pre_model/bert_config.json')
tokenizer = AutoTokenizer.from_pretrained(r'/Users/maxiong/Workpace/Code/transformers/pre_model/bert_config.json')
model = AutoModel.from_pretrained(r'/Users/maxiong/Workpace/Code/transformers/pre_model/bert_model.ckpt.index', from_tf=True, config=config)
`
this bert model is from google official chinese bert base model, in your answer, the model can be used with this libary, which was trained by this library, if that, if I want use own pretraning bert model, how to use it with this library
Problem with running Transformer Notebook: How to train a language model
Author: giulianobertoti
Pull request: no

Hello,

in the page:

https://github.com/huggingface/transformers/blob/master/notebooks/README.md

I clicked "open in colab" on the notebook "How to train a language model".

When the last run cell was run:

%%time
!{cmd}

The following error was presented:

Traceback (most recent call last):
  File "run_language_modeling.py", line 40, in <module>
    from transformers import (
ImportError: cannot import name 'CONFIG_MAPPING'
CPU times: user 42.2 ms, sys: 18 ms, total: 60.2 ms
Wall time: 5.82 s

Any suggestion?

Thank you!


Comments:
Importing CONFIG_MAPPING from https://github.com/huggingface/transformers/blob/master/src/transformers/configuration_auto.py to https://github.com/huggingface/transformers/blob/master/src/transformers/__init__.py fixes the problem (for the last version of the package)

This should be fixed on master by f8823bad9a23f6623e91e71719e65342de877cb9. Can you please try again, and re-open if necessary?

(in a Colab notebook, you'll need to re-download the `run_language_modeling.py` script using `!wget https://raw.githubusercontent.com/huggingface/transformers/master/examples/run_language_modeling.py`)

It is now showing the following error,

Traceback (most recent call last):
  File "run_language_modeling.py", line 782, in <module>
    main()
  File "run_language_modeling.py", line 677, in main
    config = AutoConfig.from_pretrained(args.config_name, cache_dir=args.cache_dir)
  File "/usr/local/lib/python3.6/dist-packages/transformers/configuration_auto.py", line 198, in from_pretrained
    "in its name: {}".format(pretrained_model_name_or_path, ", ".join(CONFIG_MAPPING.keys()))
ValueError: Unrecognized model in /content/models/RoBERTa_GPT/. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: t5, distilbert, albert, camembert, xlm-roberta, bart, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl
CPU times: user 41.1 ms, sys: 8.81 ms, total: 49.9 ms
Wall time: 6.8 s

@Yamantaka01 Your config.json in `/content/models/RoBERTa_GPT/config.json` should contain a model_type key
Confusion in understanding the output of BERTforTokenClassification class from Transformers library
Author: ghost
Pull request: no

It is the example given in the documentation of transformers pytorch library
```
from transformers import BertTokenizer, BertForTokenClassification
import torch

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertForTokenClassification.from_pretrained('bert-base-uncased')

input_ids = torch.tensor(tokenizer.encode("Hello, my dog is cute", add_special_tokens=True)).unsqueeze(0)  # Batch size 1
labels = torch.tensor([1] * input_ids.size(1)).unsqueeze(0)  # Batch size 1
outputs = model(input_ids, labels=labels)

loss, scores, hidden_states,attentions = outputs
```
Here hidden_states is a tuple of length 13 and contains hidden-states of the model at the output of each layer plus the initial embedding outputs. **I would like to know, whether hidden_states[0] or hidden_states[12] represent the final hidden state vectors**?

Thanks in advance  @thomwolf  @nreimers

Comments:
AFAIK, `12` does

For detailed explanation for this, refer
https://stackoverflow.com/questions/60847291/confusion-in-understanding-the-output-of-bertfortokenclassification-class-from-t
always occur error:AssertionError
Author: jiangjiaqi6
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

<!-- A clear and concise description of the question. -->
I put the wiki.train.raw and the wiki.test.raw in /dataset,
then run the command:
python run_lm_finetuning.py     --output_dir=output     --model_type=roberta     --model_name_or_path=roberta-base     --do_train     --train_data_file=/dataset/wiki.train.raw     --do_eval     --eval_data_file=/dataset/wiki.test.raw     --mlm

errors:
Traceback (most recent call last):
  File "run_lm_finetuning.py", line 717, in <module>
    main()
  File "run_lm_finetuning.py", line 662, in main
    train_dataset = load_and_cache_examples(args, tokenizer, evaluate=False)
  File "run_lm_finetuning.py", line 127, in load_and_cache_examples
    block_size=args.block_size,
  File "run_lm_finetuning.py", line 86, in __init__
    assert os.path.isfile(file_path)
AssertionError 

then I find the code in run_lm_finetuning.py, 
I don't know change file_path in def __init__:
def __init__(self, tokenizer, args, file_path="train", block_size=512):
I hope you can help me out.

Comments:
It seems like the paths to your data files are incorrect.

Are you sure they're not at `./dataset/wiki.train.raw ` (notice the leading `.`)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Updated/added model cards
Author: traviemcg
Pull request: yes
Labels: name: model card

Trained four models, adding/updating model cards to make consistent!

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3435?src=pr&el=h1) Report
> Merging [#3435](https://codecov.io/gh/huggingface/transformers/pull/3435?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/ccbe839ee0b78a17e74dab218bfae7efe904ac3b&el=desc) will **increase** coverage by `0.04%`.
> The diff coverage is `88.88%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3435/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3435?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3435      +/-   ##
==========================================
+ Coverage   77.56%   77.60%   +0.04%     
==========================================
  Files         100      100              
  Lines       16970    16967       -3     
==========================================
+ Hits        13162    13167       +5     
+ Misses       3808     3800       -8     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3435?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/data/processors/utils.py](https://codecov.io/gh/huggingface/transformers/pull/3435/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9kYXRhL3Byb2Nlc3NvcnMvdXRpbHMucHk=) | `24.68% <88.88%> (+2.94%)` | :arrow_up: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3435/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.85% <0.00%> (+0.13%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3435?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3435?src=pr&el=footer). Last update [83272a3...d486459](https://codecov.io/gh/huggingface/transformers/pull/3435?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).