noamaanMulla-03/transformers-issues huggingface dataset:
Add model card
Author: mrm8488
Pull request: yes
Labels: name: model card


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3658?src=pr&el=h1) Report
> Merging [#3658](https://codecov.io/gh/huggingface/transformers/pull/3658?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/39a34cc375ed79d18888464289b83713fc20f7d4&el=desc) will **decrease** coverage by `0.05%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3658/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3658?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3658      +/-   ##
==========================================
- Coverage   78.29%   78.23%   -0.06%     
==========================================
  Files         104      104              
  Lines       17628    17628              
==========================================
- Hits        13801    13791      -10     
- Misses       3827     3837      +10     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3658?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/tokenization\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3658/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdXRpbHMucHk=) | `90.65% <0.00%> (-0.84%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3658/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `92.63% <0.00%> (-0.66%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3658?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3658?src=pr&el=footer). Last update [39a34cc...f390cb4](https://codecov.io/gh/huggingface/transformers/pull/3658?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Add model card
Author: mrm8488
Pull request: yes
Labels: name: model card


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3659?src=pr&el=h1) Report
> Merging [#3659](https://codecov.io/gh/huggingface/transformers/pull/3659?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/39a34cc375ed79d18888464289b83713fc20f7d4&el=desc) will **not change** coverage by `%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3659/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3659?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #3659   +/-   ##
=======================================
  Coverage   78.29%   78.29%           
=======================================
  Files         104      104           
  Lines       17628    17628           
=======================================
  Hits        13801    13801           
  Misses       3827     3827           
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3659?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3659/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `93.28% <0.00%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3659?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3659?src=pr&el=footer). Last update [39a34cc...8d1de79](https://codecov.io/gh/huggingface/transformers/pull/3659?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Fix RoBERTa/XLNet Pad Token in run_multiple_choice.py
Author: ethanjperez
Pull request: yes

`convert_examples_to_fes atures` sets `pad_token=0` by default, which is correct for BERT but incorrect for RoBERTa (`pad_token=1`) and XLNet (`pad_token=5`). I think the other arguments to `convert_examples_to_features` are correct, but it might be helpful if someone checked who is more familiar with this part of the codebase.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3631?src=pr&el=h1) Report
> Merging [#3631](https://codecov.io/gh/huggingface/transformers/pull/3631?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/243e687be6cd701722cce050005a2181e78a08a8&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3631/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3631?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3631      +/-   ##
==========================================
- Coverage   78.30%   78.28%   -0.02%     
==========================================
  Files         104      104              
  Lines       17627    17627              
==========================================
- Hits        13802    13800       -2     
- Misses       3825     3827       +2     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3631?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3631/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `93.45% <0.00%> (-0.17%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3631/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.97% <0.00%> (-0.13%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3631?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3631?src=pr&el=footer). Last update [243e687...284fa1b](https://codecov.io/gh/huggingface/transformers/pull/3631?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Which model is best to used for language model rescoring for ASR
Author: LearnedVector
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

<!-- A clear and concise description of the question. -->
Hello all, I am wanting to use this library to rescore an output from an automatic speech recognition model. Still learning a lot about language model so out of curiosity for anyone who's tried, which model has given you the best performance? Looking to a language model that can predict score the probability of a sentence most effectively. 

Comments:
Same as https://github.com/google-research/bert/issues/35

And https://github.com/huggingface/transformers/issues/37

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


GPT2 can be used for rescoring https://arxiv.org/pdf/1910.11450.pdf

[I tested](https://github.com/simonepri/lm-scorer/blob/master/tests/models/test_gpt2.py#L52-L239) GPT2  on different english error types (I used the one defined in the [ERRANT framework](https://www.aclweb.org/anthology/P17-1074/)) and it seems that is able to give a lower probability to the wrong version of a sentence (At least for simple examples).
How to add a fc classification head to BertForQA to make a MTL-BertForQA model?
Author: LukasMut
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**: 

Comments:
I am currently trying to figure out how to add an additional classification head to the BertForQA model. I am not sure which is the best / most efficient way to do that. Should I rewrite the source code for BertForQA and inherit from BertPretrainedModel, or should I rather inherit from BertForQA and change the forward pass? The forward pass needs to be fairly similar though... Any help is appreciated!

This is a matter of taste. I would write my own class which inherits from BertModel but this is completely up to you.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Pipeline error when creating a model without a model card json file (on Windows)
Author: AlecS12
Pull request: no
Labels: name: wontfix

name: Core: Pipeline

## 🐛 Bug

<!-- Important information -->

Model I am using (Bert, XLNet....):
Bert
Language I am using the model on (English, Chinese....):
English
The problem arise when using:
* [ ] the official example scripts: (give details)
* [x] my own modified scripts: (give details)
```python
import sys
from transformers import pipeline

if __name__ == '__main__':
    print("start")
    nlp_ft = pipeline('question-answering', model=r'C:\Users\a652726\PycharmProjects\src\data\raw\qa\wwm-bert-uncased-finetuned-squad',
                      tokenizer='bert-large-uncased')
```

Results in the ValueError: no modelcard.json file (which i do not have)
My  fix (hack):
In modelcard.py replace (line 164):

```python
        except EnvironmentError:
            if pretrained_model_name_or_path in ALL_PRETRAINED_CONFIG_ARCHIVE_MAP:
```
with
```python
        except (EnvironmentError, ValueError):
            if pretrained_model_name_or_path in ALL_PRETRAINED_CONFIG_ARCHIVE_MAP:
```
This results in:

```python
            logger.warning("Creating an empty model card.")
```
And everything works fine after this.

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x ] my own task or dataset: (give details)
 Changing QA pipeline to work on a fixed set of spans (like multiple choice QA task, or classification)

## To Reproduce

Steps to reproduce the behavior:

1.
2.
3.

<!-- If you have a code sample, error messages, stack traces, please provide it here as well. -->

## Expected behavior

<!-- A clear and concise description of what you expected to happen. -->

## Environment

* OS:  Windows
* Python version: 3.7
* PyTorch version: 1.3
* PyTorch Transformers version (or branch): master (post 2.3)
* Using GPU ? no
* Distributed or parallel setup ? no
* Any other relevant information:

## Additional context

<!-- Add any other context about the problem here. -->


Comments:
Please format your post correctly by using code blocks. https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
German Bert tokenizer does not recognize (some) special characters (!,?,...)
Author: andrey999333
Pull request: no
Labels: name: wontfix

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):

**Bert**

Language I am using the model on (English, Chinese ...):

**German**

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ X] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ X] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

`tokenizer = BertTokenizer.from_pretrained('bert-base-german-cased')`
`tokenizer.encode("Hallo!", add_special_tokens=False)`

returns:

`[5850, 26910, 2]`

where `2` is the index of '[UNK]' (unknown) token. Same would happen with 'Hallo?'. In the vocab.txt we have tokens '##!' and '##?' but not '?' or '!'. On the other hand some special characters are recognized, like ':' or ';'.

## Expected behavior

At least such common characters as '!' and '?' should be recognized by tokenizer


## Environment

* OS: Ubuntu
* Python version: 3.7
* `transformers` version (or branch): 2.3

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Sounds like this was fixed by @Timoeller in #3618, @andrey999333 

Thanks for referencing. This bug should be fixed with the changes we applied to the vocabulary.
Find more infos in the separate issue here: deepset-ai/FARM/issues/60
TFCamembertModel
Author: jkintzinger
Pull request: no
Labels: name: wontfix

Hi, 

I wanted to use the tensorflow version of Camembert : TFCamembertModel, but the implememtation is not available with the v2.3.0 version : https://huggingface.co/transformers/v2.3.0/model_doc/camembert.html. 

But TFCamembertModel  seems to be available with another version of transformers : https://huggingface.co/transformers/model_doc/camembert.html.

Is this a new or old version of the library ?

Any way, have you already succeeded importing TFCamembertModel ?

Thanks a lot !



Comments:
Hi, the CamemBERT model for tensorflow was merged yesterday, and is therefore available from the master branch right now. You can install it with 

```py
pip install git+https://github.com/huggingface/transformers
```

It will be in the next transformers release (2.4.0 or 2.3.1), which should be released later today.

Awesome, thank you !

Hi again @LysandreJik ,

I have checked the new version of Transformers and an error occurred when I tried to load TFCamembertModel:

`TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType`

I think I found what was missing. The pretrained models are not available, I mean the list is empty when I try to load a pretrained model.

`OSError: Model name 'test_list' was not found in model name list. We assumed 'https://s3.amazonaws.com/models.huggingface.co/bert/test_list/config.json' was a path, a model identifier, or url to a configuration file named config.json or a directory containing such a file but couldn't find any such file at this path or url.`

So I checked the pretrained model archives avaible in "modeling_tf_camembert.py" and, indeed, the dictionnary is empty :

`TF_CAMEMBERT_PRETRAINED_MODEL_ARCHIVE_MAP = {}`

Is it on purpose ?

Thanks


Hi, indeed there are not checkpoints in the archive map. The contributor (@jplu) that contributed the tensorflow architecture has uploaded checkpoints that you can use: https://huggingface.co/jplu/tf-camembert-base.

We're currently working on our website so that it better reflects the following:
- The CamemBERT model has official weights that are usable in PyTorch, but do not currently have any TensorFlow equivalent
- There are community models (cf. jplu/tf-camembert-base) which can be used instead.

We're trying to make sure that contributing models is easy and that weights are easily identifiable for users. This is still a work in progress.

cc @julien-c @joshchagani 

TLDR: in transformers v2.4.0, the following should work:

```py
from transformers import TFCamembertModel
model = TFCamembertModel.from_pretrained("jplu/tf-camembert-base")
```

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
[WIP] Remove tokenizers dependency
Author: BramVanroy
Pull request: yes

In v2.5.1 the tokenizers don't default to the fast implementations (as introduced in v2.5.0), which I think is a good thing when looking at the issues that have arisen from it. Even though the performance of the new tokenizers is phenomenal (and I complement everyone who has been working on it!), it seems a bit premature to make `tokenizers` a dependency. (In addition, see for instance this topic concerning installation issues: https://github.com/huggingface/transformers/issues/2980.)

Even though the fast implementation isn't the default any more, it is still part of the dependencies in setup.py. This PR removes `tokenizers` from the dependency list but indicates in the documentation that having `tokenizers` installed and using `use_fast` can result in great performance improvements.

Generally, I am not satisfied with how this PR has been implemented (quite some duplication across the different tokenizers), but I don't really see another way. Alternative ideas are definitely welcome. If, on the other hand, you decide to keep the dependency, that is fine too.

Note: my flake8 keeps failing with an obscure error so can't do the manual code checking now. Might try again later.

Note: tests will fail for the fast tokenizers (and perhaps on some other imports). I'll look further into this if you decide that this PR is welcome.

Comments:
Speedup torch summarization tests
Author: sshleifer
Pull request: yes

Speedup torch summarization tests  by using small models that are faster to download and instantiate.

Comments:
Non slow test speed before change:
![image](https://user-images.githubusercontent.com/6045025/78608104-459d8000-782e-11ea-9a30-a16ea20e3eec.png)


There is a tiny TF T5 model now as well via: 
`model = TFAutoModelWithLMHead.from_pretrained("patrickvonplaten/t5-tiny-random")`

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3663?src=pr&el=h1) Report
> Merging [#3663](https://codecov.io/gh/huggingface/transformers/pull/3663?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/0a9d09b42a9c7c1ccc00da48486a1188078e8594&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3663/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3663?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3663      +/-   ##
==========================================
- Coverage   78.03%   78.02%   -0.02%     
==========================================
  Files         104      104              
  Lines       17708    17708              
==========================================
- Hits        13819    13817       -2     
- Misses       3889     3891       +2     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3663?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3663/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `92.96% <0.00%> (-0.17%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3663/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.97% <0.00%> (-0.13%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3663?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3663?src=pr&el=footer). Last update [0a9d09b...4751188](https://codecov.io/gh/huggingface/transformers/pull/3663?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


LGTM, but FYI for context, I think the reason we were using the real models was that we intended to do integration testing: in `_test_mono_column_pipeline` we only test equality of keys but we could have tested equality (or closeness) of values.

Makes sense @julien-c .  I'd be happy to add some `@slow` integration tests and try to  fulfill the original intent
How can I track the performance of my GPT-2 model during finetuning?
Author: hmdgit
Pull request: no

Hi,

I am new in using Transformer HugginFace library. 

I am using [Google Colab](https://colab.research.google.com/github/interactive-fiction-class/interactive-fiction-class.github.io/blob/master/homeworks/language-model/hw4_transformer.ipynb) to fine tune GPT-2 model. Google Colab only displays the output of last 5000 lines. So, I could not be able to figure out the performance of previous checkpoints, whose output vanishes.

I would like to track the performance of my training model during the whole period of time. Is it possible to track it by using **tensorboard**? or is there any other way exist? I have noticed a variable "logging_steps" in [run_language_modeling.py](https://github.com/huggingface/transformers/blob/master/examples/run_language_modeling.py) file, but I donot have an idea how can I use it to track performance of training model in Google Colab?





Comments:
[run_language_modeling.py](https://github.com/huggingface/transformers/blob/master/examples/run_language_modeling.py) allow you to track you training performance (validation perplexity, training loss and learning rate) using tensorboad. You need to have tensorboad installed. And then you can just run `tensorboard --logdir=runs` to follow your training.

Thanks a lot. It works. 
I have used these commands in Google Colab
```
%load_ext tensorboard
%tensorboard --logdir=runs
```