noamaanMulla-03/transformers-issues huggingface dataset:
No CORS Policy for Write With Transformer Endpoints
Author: ivokun
Pull request: no
Labels: name: Write With Transformer

# 🐛 Bug

## Information
Write with Transformer is a very good place for trying available models. But, there is no CORS policy to secure it. This could potentially be used by an unauthorized third-parties or attackers to DDoS.

## To reproduce

Steps to reproduce the behavior:
POST request to `/autocomplete/distilgpt2/small`
```
curl -d '{"context":"See how a modern neural network auto-completes your text 🤗","model_size":"distilgpt2/small","top_p":0.9,"temperature":1,"max_time":1}' -H 'Content-Type: application/json' https://transformer.huggingface.co/autocomplete/distilgpt2/small
```
Potential endpoints:
1. `/autocomplete/distilgpt2/small`
1. `/autocomplete/gpt2/small`
1. `/autocomplete/gpt2/medium`
1. `/autocomplete/gpt2/large`
1. `/autocomplete/gpt2/xl` <- 504 error
1. `/autocomplete/pplm` <- 504 error


<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

The endpoints could be used by unauthorized third-parties and risk of DDoS .

Comments:
Hi @ivokun – this was actually on purpose as someone was using it as an API

We'll secure it down the line if it gets abused.
Create README.md
Author: benjamin-mlr
Pull request: yes
Labels: name: model card


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4062?src=pr&el=h1) Report
> Merging [#4062](https://codecov.io/gh/huggingface/transformers/pull/4062?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/6faca88ee0c472de8207e648b0999a1ee01ff127&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4062/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4062?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4062   +/-   ##
=======================================
  Coverage   78.91%   78.91%           
=======================================
  Files         114      114           
  Lines       18670    18670           
=======================================
  Hits        14734    14734           
  Misses       3936     3936           
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4062?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4062/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `92.61% <0.00%> (-0.17%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4062/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.04% <0.00%> (+0.12%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4062?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4062?src=pr&el=footer). Last update [6faca88...6f20b16](https://codecov.io/gh/huggingface/transformers/pull/4062?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Create README.md model_card camembert/camembert-base-oscar-4gb
Author: benjamin-mlr
Pull request: yes
Labels: name: model card


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4063?src=pr&el=h1) Report
> Merging [#4063](https://codecov.io/gh/huggingface/transformers/pull/4063?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/6faca88ee0c472de8207e648b0999a1ee01ff127&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4063/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4063?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4063   +/-   ##
=======================================
  Coverage   78.91%   78.91%           
=======================================
  Files         114      114           
  Lines       18670    18670           
=======================================
  Hits        14734    14734           
  Misses       3936     3936           
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4063?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4063?src=pr&el=footer). Last update [6faca88...4030892](https://codecov.io/gh/huggingface/transformers/pull/4063?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Create model_card camembert-base-ccnet-4gb
Author: benjamin-mlr
Pull request: yes
Labels: name: model card


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4064?src=pr&el=h1) Report
> Merging [#4064](https://codecov.io/gh/huggingface/transformers/pull/4064?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/6faca88ee0c472de8207e648b0999a1ee01ff127&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4064/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4064?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4064   +/-   ##
=======================================
  Coverage   78.91%   78.92%           
=======================================
  Files         114      114           
  Lines       18670    18670           
=======================================
+ Hits        14734    14735    +1     
+ Misses       3936     3935    -1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4064?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4064/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.55% <0.00%> (+0.41%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4064?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4064?src=pr&el=footer). Last update [6faca88...0c16603](https://codecov.io/gh/huggingface/transformers/pull/4064?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Create model_card camembert-base-ccnet
Author: benjamin-mlr
Pull request: yes
Labels: name: model card


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4065?src=pr&el=h1) Report
> Merging [#4065](https://codecov.io/gh/huggingface/transformers/pull/4065?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/6faca88ee0c472de8207e648b0999a1ee01ff127&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4065/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4065?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4065   +/-   ##
=======================================
  Coverage   78.91%   78.91%           
=======================================
  Files         114      114           
  Lines       18670    18670           
=======================================
  Hits        14734    14734           
  Misses       3936     3936           
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4065?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4065?src=pr&el=footer). Last update [6faca88...5f06af6](https://codecov.io/gh/huggingface/transformers/pull/4065?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
create model_card camembert-base-wikipedia-4gb
Author: benjamin-mlr
Pull request: yes
Labels: name: model card


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4066?src=pr&el=h1) Report
> Merging [#4066](https://codecov.io/gh/huggingface/transformers/pull/4066?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/6faca88ee0c472de8207e648b0999a1ee01ff127&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4066/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4066?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4066   +/-   ##
=======================================
  Coverage   78.91%   78.92%           
=======================================
  Files         114      114           
  Lines       18670    18670           
=======================================
+ Hits        14734    14735    +1     
+ Misses       3936     3935    -1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4066?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4066/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `92.93% <0.00%> (+0.16%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4066?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4066?src=pr&el=footer). Last update [6faca88...2af7f81](https://codecov.io/gh/huggingface/transformers/pull/4066?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Model Card: gaochangkuan  README.md
Author: ScottishFold007
Pull request: yes
Labels: name: model card


Comments:
As discussed over email, can you move this file to https://huggingface.co/gaochangkuan/model_dir ?

Thanks!

> As discussed over email, can you move this file to https://huggingface.co/gaochangkuan/model_dir ?
> 
> Thanks!

Ok， I will do it later~

I meant the file path of the README.md file :)

I've done it on your branch directly. Thank you for sharing: [model page](https://huggingface.co/gaochangkuan/model_dir).
Continue training args and tqdm in notebooks
Author: parmarsuraj99
Pull request: yes

Added a little more description to the metadata of training args for new scripts. 
`--overwrite_output_dir` can be used to continue training from checkpoint if `--output_dir` points to a directory already having checkpoints.

This can help reduce the confusion when migrating from older script.


tqdm in colab prints every step in new line.
so updated to 
`from tqdm.auto import tqdm`

Comments:
Updates committed

Thanks!
BertEncoder head_mask not subscript-able error when not passed
Author: apsdehal
Pull request: no
Labels: name: wontfix

## 🐛 Bug

BertEncoder takes in head_mask parameters with default value of None but on https://github.com/huggingface/pytorch-transformers/blob/master/pytorch_transformers/modeling_bert.py#L431, ith index is checked without checking if head_mask is None. If by default nothing is passed this results in error.

## Fix
Check head_mask is not None in https://github.com/huggingface/pytorch-transformers/blob/master/pytorch_transformers/modeling_bert.py#L431

Comments:
Got same issue. If no `head_mask` is given to `.forward` method of `BertEncoder`, then the following code will cause `TypeError: 'NoneType' object is not subscriptable`.

https://github.com/huggingface/transformers/blob/a701c9b32126f1e6974d9fcb3a5c3700527d8559/transformers/modeling_bert.py#L348

I wonder if there will be a case of using `BertEncoder` without any `head_mask`. Though, this issue should be addressed by checking whether `head_mask` is None, and expand them as size `[config.num_hidden_layers]`.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


As bot closed the issue and it is still existing, I provided a fix it in the PR linked above.
[gitignore] fixtures created by unit tests
Author: sshleifer
Pull request: yes


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4031?src=pr&el=h1) Report
> Merging [#4031](https://codecov.io/gh/huggingface/transformers/pull/4031?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/4e817ff41885063e08bb3bcd63e5adfd835b9911&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4031/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4031?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4031   +/-   ##
=======================================
  Coverage   78.44%   78.44%           
=======================================
  Files         111      111           
  Lines       18518    18518           
=======================================
  Hits        14527    14527           
  Misses       3991     3991           
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4031?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4031/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `68.49% <0.00%> (-0.37%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4031/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.06% <0.00%> (+0.12%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4031?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4031?src=pr&el=footer). Last update [4e817ff...361724e](https://codecov.io/gh/huggingface/transformers/pull/4031?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Which tests yield those fixtures? I just ran all the tests and I have none of those

It comes from adding a failing test so we don't need it. My bad.
[testing] add timeout_decorator
Author: sshleifer
Pull request: yes

This is a simple way to make sure code doesn't get slower over time. Since it is a new dependency, I wanted to show a tiny PR before I use it more.

Comments:
Feels like it could be reimplemented in a few lines of code – do we need to add a new dependency for this?

I copy pasted it. Would love to understand more about the adding a dependency vs maintaining code tradeoff!

I think that's a great addition! Trying to keep the tests short is very important I think :-) 

This is too long to copy/paste, so I'd see two options:
- add it as a dependency to extras["testing"]
- take just the signals based implem, clean it up/distill it down to a few (10) lines of code and add it in `tests/`

I think we should do `extras['testing']` (that was my first attempt on this PR).
If we delete the `signals=False` logic, I don't think circleci will work in distributed testing mode.

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3543?src=pr&el=h1) Report
> Merging [#3543](https://codecov.io/gh/huggingface/transformers/pull/3543?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/b8686174be75220d2c26a961597a39ef4921b616&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3543/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3543?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3543      +/-   ##
==========================================
+ Coverage   78.84%   78.85%   +0.01%     
==========================================
  Files         114      114              
  Lines       18691    18691              
==========================================
+ Hits        14737    14739       +2     
+ Misses       3954     3952       -2     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3543?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3543/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `92.93% <0.00%> (+0.16%)` | :arrow_up: |
| [src/transformers/trainer.py](https://codecov.io/gh/huggingface/transformers/pull/3543/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmVyLnB5) | `43.90% <0.00%> (+0.34%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3543?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3543?src=pr&el=footer). Last update [b868617...8b22919](https://codecov.io/gh/huggingface/transformers/pull/3543?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
How to using vocab other language in run_ner.py?
Author: TorRient
Pull request: no

Help me, thank you!!!

Comments:
Which language do you want to use?
You can load a model that pre-trained with a certain language or you can use mBERT / XLM instead.

> Which language do you want to use?
> You can load a model that pre-trained with a certain language or you can use mBERT / XLM instead.

Thank for reply
i want to use BERT for Vietnamese language

Unfortunately, there is no BERT for Vietnamese available for now.
You will have to wait for somebody to pre-trained the Vietnamese model or you can use Multilingual-BERT (mBERT) instead:
```bert-base-multilingual-cased``` or ```bert-base-multilingual-uncased```

These models were pre-trained with [top 100 languages in Wikipedia](https://meta.wikimedia.org/wiki/List_of_Wikipedias).
[ci] Load pretrained models into the default (long-lived) cache
Author: julien-c
Pull request: yes

There's an inconsistency right now where:
- we load some models into CACHE_DIR
- and some models in the default cache
- and often, in both for the same models

When running the RUN_SLOW tests, this takes a lot of disk space, time, and bandwidth.

I'd rather always use the default cache


Comments:
Sorry I missed this. This is cool!
Proposal: saner num_labels in configs.
Author: julien-c
Pull request: yes

See https://github.com/guillaume-be/rust-bert/pull/21 for more context

Comments:
I like this better as well. Probably won't be backwards compatible with users that have their config with `_num_labels` though.

Like it better as well! Agree with @LysandreJik that some configs might need to be fixed manually

Well, `_num_labels` (if present) should always be consistent with the length of `id2label`, no?

You're right, it should!

All right, merging this. @patrickvonplaten if you want to re-run your cleaning script at some point feel free to do it :) (let me know before)
Why is masking ID optional?
Author: tqdo
Pull request: no

I see many models have masking ID as an optional input. To my understanding, the models need to know which tokens are padded to avoid doing attention with those tokens. How can the models figure this out if we do not provide masking ID? Do these models consider tokens with id 0 to be the masked tokens?

Comments:
In my past experience, I noticed that masked tokens would get very low attention scores(that's what self attention's supposed to do) and would not contribute to the model's accuracy. I wonder how would it be in a transformer architecture.

Not all models requires this feature, mainly Roberta so it includes XLM-R, Camembert, etc.

I think your question is why are the `attention_mask` model inputs optional?
This is because you need attention masks only when padding. If you're not batching your inputs, you do not need to pad and therefore do not need to specify an attention mask.

@LysandreJik I wonder if there is padding and the user does not provide `attention_mask`, does the model think there is no padding or there is a mechanism to figure it out uder the hood (maybe treat token ID 0 as padded token). I am just trying to fully understand the model's behavior. Thanks

No, if you don't provide any `attention_mask` then the model will attend to every token the same way. You absolutely should use the `attention_mask` as soon as you're padding, or the model will output flawed results!

Got it, thanks a lot.
GePpeTto!
Author: LoreDema
Pull request: no
Labels: name: New model

# 🌟 New model addition

## Model description

GePpeTto is a GPT-2 model for italian. For further details checkout the paper: https://arxiv.org/abs/2004.14253

## Open source status

* [X ] the model implementation is available: we used huggingface's GPT-2 implementation.
* [X ] the model weights are available: you can find the model here: https://github.com/LoreDema/GePpeTto
* [X ] who are the authors: @LoreDema @michelecafagna26 @malvinanissim @Falice1977 @marcoguerini


Comments:
Awesome! These instructions might help: 

https://github.com/huggingface/transformers#Quick-tour-of-model-sharing

Hi @LoreDema, thanks for sharing GePpeTto, it's really cool!

Yes you can juste create a user account and an organization on huggingface.co and share the weights there. (and add a model card here if you can)

Awesome, just pushed the model and created a pull request for the card https://github.com/huggingface/transformers/pull/4099