noamaanMulla-03/transformers-issues huggingface dataset:
Cannot load pretrained model from repo.
Author: jordaniac89
Pull request: no

Hi, I'm trying to load a model from the repository:

```
tokenizer = AutoTokenizer.from_pretrained("NeuML/bert-small-cord19-squad2")
model = AutoModelForQuestionAnswering.from_pretrained("NeuML/bert-small-cord19-squad2")
```
but I'm receiving the error:
`OSError: Model name 'NeuML/bert-small-cord19-squad2' was not found in tokenizers model name list (bert-base-uncased, bert-large-uncased, bert-base-cased, bert-large-cased, bert-base-multilingual-uncased, bert-base-multilingual-cased, bert-base-chinese, bert-base-german-cased, bert-large-uncased-whole-word-masking, bert-large-cased-whole-word-masking, bert-large-uncased-whole-word-masking-finetuned-squad, bert-large-cased-whole-word-masking-finetuned-squad, bert-base-cased-finetuned-mrpc, bert-base-german-dbmdz-cased, bert-base-german-dbmdz-uncased). We assumed 'NeuML/bert-small-cord19-squad2' was a path or url to a directory containing vocabulary files named ['vocab.txt'] but couldn't find such vocabulary files at this path or url.
`

I've tried a few other models with the same result. The vocab.txt file is there in each of them. Am I missing something?


EDIT: Current libraries -
beautifulsoup4==4.9.1
boto3==1.10.30
botocore==1.13.30
certifi==2019.11.28
chardet==3.0.4
Click==7.0
docutils==0.15.2
fsspec==0.6.1
future==0.18.2
html2text==2020.1.16
idna==2.8
jmespath==0.9.4
joblib==0.14.0
nltk==3.4.5
numpy==1.17.4
pandas==0.25.3
Pillow==7.1.2
python-dateutil==2.8.0
pytz==2019.3
regex==2019.11.1
requests==2.22.0
s3fs==0.4.0
s3transfer==0.2.1
sacremoses==0.0.35
scikit-learn==0.21.3
scipy==1.3.2
sentencepiece==0.1.83
six==1.13.0
soupsieve==2.0.1
torch==1.5.0
torchvision==0.6.0
tqdm==4.40.0
transformers==2.2.0
urllib3==1.25.7
wikipedia==1.4.0

Comments:
Hello @jordaniac89  I just tried this, and it worked. Can you try this again ?

Yeah, nothing. My assumption was that from_pretrained() should reach out to s3 and download the model if it can't find it?

Figured it out. Just an fyi that `pip install transformers` installs v. 2.2.0. I needed to specify 2.5.1 with `pip install transformers==2.5.1`

Glad you could solve the issue!
Fine-tuning of RoBERTa
Author: Aktsvigun
Pull request: no

Good afternoon,
I'm trying to fine-tune RoBERTa on my own dataset, following the instruction provided here [https://huggingface.co/transformers/v1.2.0/examples.html](url). However, I cannot find the file `run_lm_finetuning.py` - could you please clarify, is the instruction valid or the file has really been deleted? Thanks in advance!

Be safe,
Akim



Comments:
Hello @Aktsvigun 
To run the examples you'll need to clone the transformer repo. All examples can be found in examples directory.  You can find `run_lm_finetuning` here.
run_language_modeling.py
https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_language_modeling.py

@patil-suraj  thank you for your response,
feel a bit dummy, is the function name changed to run_language_modeling? 😹
Cannot find the name `run_lm_finetuning` on the page itself.
<img width="1228" alt="Снимок экрана 2020-06-04 в 16 57 00" src="https://user-images.githubusercontent.com/36672861/83765910-6b0b0680-a684-11ea-8ff6-6ba29f833870.png">

Be safe,
Akim


yes, the filename `run_lm_finetuning` is changed to `run_language_modeling.py`

Hi @Aktsvigun, the documentation you're linking is for `transformers` v1.2.0. If you want to run `v1.2.0` scripts, you should look at the [tag v1.2.0](https://github.com/huggingface/transformers/tree/1.2.0/examples).

Please note that the latest scripts are more stable. As @patil-suraj said, the script you're looking for was renamed, as you can see in the [current documentation](https://huggingface.co/transformers/examples.html).
Create README.md
Author: orena1
Pull request: yes
Labels: name: model card

Main issue is to refer to the fairseq website.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4743?src=pr&el=h1) Report
> Merging [#4743](https://codecov.io/gh/huggingface/transformers/pull/4743?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/1b5820a56540a2096daeb43a0cd8247c8c94a719&el=desc) will **increase** coverage by `0.22%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4743/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4743?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4743      +/-   ##
==========================================
+ Coverage   77.11%   77.34%   +0.22%     
==========================================
  Files         128      128              
  Lines       21061    21061              
==========================================
+ Hits        16242    16290      +48     
+ Misses       4819     4771      -48     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4743?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4743/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `88.63% <0.00%> (-0.17%)` | :arrow_down: |
| [src/transformers/trainer.py](https://codecov.io/gh/huggingface/transformers/pull/4743/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmVyLnB5) | `39.62% <0.00%> (+0.23%)` | :arrow_up: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4743/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `90.15% <0.00%> (+0.24%)` | :arrow_up: |
| [src/transformers/modeling\_gpt2.py](https://codecov.io/gh/huggingface/transformers/pull/4743/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19ncHQyLnB5) | `86.39% <0.00%> (+14.55%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4743?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4743?src=pr&el=footer). Last update [1b5820a...24a7fe1](https://codecov.io/gh/huggingface/transformers/pull/4743?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Create model card for T5-base fine-tuned for Sentiment Span Extraction
Author: mrm8488
Pull request: yes
Labels: name: model card


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4737?src=pr&el=h1) Report
> Merging [#4737](https://codecov.io/gh/huggingface/transformers/pull/4737?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/3e5928c57d57db3071638e6beaec9349a75b6a22&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4737/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4737?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4737   +/-   ##
=======================================
  Coverage   77.29%   77.29%           
=======================================
  Files         128      128           
  Lines       21004    21004           
=======================================
  Hits        16234    16234           
  Misses       4770     4770           
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4737?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4737/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.85% <0.00%> (-0.42%)` | :arrow_down: |
| [src/transformers/trainer.py](https://codecov.io/gh/huggingface/transformers/pull/4737/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmVyLnB5) | `39.62% <0.00%> (+0.23%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4737?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4737?src=pr&el=footer). Last update [3e5928c...d41d001](https://codecov.io/gh/huggingface/transformers/pull/4737?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Thanks Manuel:)

My pleasure. Coming soon model card for https://huggingface.co/mrm8488/t5-base-finetuned-summarize-news and RuPERTa ofc 😉
GPT2TokenizerFast raises pad_token error even if not used
Author: tomhosking
Pull request: no

# 🐛 Bug

## Information

Model: GPT2

Language: English

Encoding with the `GPT2TokenizerFast` causes a `pad_token` error to be sent to stderr, despite not attempting to access that property.

## To reproduce

```import transformers
from transformers import GPT2TokenizerFast

tokenizer = GPT2TokenizerFast.from_pretrained('gpt2')

res = tokenizer.encode("This is a sentence")

print(transformers.__version__)
```

Output:
```
Using pad_token, but it is not set yet.
2.11.0
```

## Expected behavior

I'm aware that GPT-2 doesn't include a pad token (#2630) - I haven't tried to use it. I would expect no error to be displayed until I try to access that property.

## Environment info

- `transformers` version: 2.11.0
- Platform: Ubuntu
- Python version: 3.6.9
- PyTorch version (GPU?): 1.5.0 with CUDA
- Tensorflow version (GPU?): n/a
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no


Comments:
I can reproduce and think I've found the cause, working on this.
removed deprecated use of Variable API from pplm example
Author: prajjwal1
Pull request: yes

This is same as previous [PR](https://github.com/huggingface/transformers/pull/4156) which I closed due to code styling issue. Didn't know that specific isort version was supposed to be used.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4619?src=pr&el=h1) Report
> Merging [#4619](https://codecov.io/gh/huggingface/transformers/pull/4619?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/842588c12ffbbe3502c5ab4a18646ad31d9c1e34&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4619/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4619?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4619   +/-   ##
=======================================
  Coverage   78.02%   78.03%           
=======================================
  Files         124      124           
  Lines       20626    20626           
=======================================
+ Hits        16093    16095    +2     
+ Misses       4533     4531    -2     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4619?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4619/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `90.53% <0.00%> (+0.11%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4619/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `88.66% <0.00%> (+0.16%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4619?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4619?src=pr&el=footer). Last update [842588c...8588578](https://codecov.io/gh/huggingface/transformers/pull/4619?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


@julien-c  Can you please see this ?

@sgugger Can you please see it ?
PPLM Write With Transformer demo not working 
Author: songproducer
Pull request: no

# 🐛 Bug

## Information

Model I am using: PPLM

Language I am using the model on (English, Chinese ...): English 

The problem arises when using:
* [ ] the official example scripts: (give details below)
ios Safari and Firefox 

The tasks I am working on is:
Trying out the demo
## To reproduce

Steps to reproduce the behavior:

1.
2.
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version:
- Platform:
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:


Comments:
cc @julien-c 

See https://github.com/huggingface/transformers/issues/4661#issuecomment-636911923
NER: Add new WNUT’17 example
Author: stefan-it
Pull request: yes
Labels: name: Ex: Named Entity Recognition

name: Documentation

Hi,

this PR extends the NER example section, and adds an extra section for fine-tuning a NER model on the (difficult) WNUT’17 shared task:

> The WNUT’17 shared task focuses on identifying unusual, previously-unseen entities in the context of emerging discussions.
> Named entities form the basis of many modern approaches to other tasks (like event clustering and summarization), but recall on
> them is a real problem in noisy text - even among annotators. This drop tends to be due to novel entities and surface forms.

I also added my pre-processing script, that splits longer sentences into smaller ones (once the max. subtoken length is reached).

Comments:
Looks great!

(I've relaunched the failing CI test that's unrelated)

@julien-c do you think rebasing onto latest master would fix that problem?
Can't save DistilBert model.
Author: sainimohit23
Pull request: no
Labels: name: wontfix

Model:
```
input_layer = tf.keras.layers.Input(shape = (attention_mask.shape[1],), dtype='int64')
bert = TFDistilBertModel.from_pretrained("distilbert-base-cased")(input_layer)
bert = bert[0][:,0,:]              
bert = tf.keras.layers.Dense(units=20, activation='relu')(bert)
classifier = tf.keras.layers.Dense(units=train_y.shape[1], activation='softmax')(bert)
model = tf.keras.models.Model(inputs=input_layer, outputs=classifier)
model.summary()
```

After training when I try to save the model using:
```
# serialize model to JSON
model_json = model.to_json()
with open("model.json", "w") as json_file:
    json_file.write(model_json)
# serialize weights to HDF5
model.save_weights("model.h5")
print("Saved model to disk")
```

I am getting this error:
```
File "train_DistilBERT_model.py", line 138, in <module>
    model_json = model.to_json()
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/network.py", line 1254, in to_json
    model_config = self._updated_config()
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/network.py", line 1232, in _updated_config
    config = self.get_config()
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/network.py", line 918, in get_config
    return copy.deepcopy(get_network_config(self))
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/network.py", line 1993, in get_network_config
    layer_config = serialize_layer_fn(layer)
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/tensorflow_core/python/keras/utils/generic_utils.py", line 198, in serialize_keras_object
    config = instance.get_config()
  File "/home/v-mohit.saini/anaconda3/envs/flask/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/network.py", line 917, in get_config
    raise NotImplementedError
NotImplementedError
```

How to fix this?

Comments:
Could you provide all the information related to your environment as the bug template recommends?

Can you try installing from master? In the latest pypi version, we only handled the `save_pretrained` method, not the `save`/`save_weights` methods. This should have been changed with #3103.

The changes in #3103 only address serialization of `TF*MainLayer` classes, used within a general Functional/Sequential API Keras model (which was my use case). Looking at [the Network docstring](https://github.com/tensorflow/tensorflow/blob/5c4931bbf69e0f006f210c6382a234e83dd4dc8e/tensorflow/python/keras/engine/network.py#L89-L97) it seems like the `TF*Model` classes, being “subclass models”, need more work in order to support Keras serialization.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
how to get activation weights of a pretrained model?
Author: jmamou
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->
how to get activation weights of a pretrained model?

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**: 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Create model card for tblard/allocine
Author: TheophileBlard
Pull request: yes
Labels: name: model card

Model card for: https://huggingface.co/tblard/tf-allocine

This is a french sentiment analysis model, trained from camembert-base, finetuned on Allociné.fr data.

Original repo: https://github.com/TheophileBlard/french-sentiment-analysis-with-bert

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4775?src=pr&el=h1) Report
> Merging [#4775](https://codecov.io/gh/huggingface/transformers/pull/4775?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/17a88d31925a9308e4d7275420033f07a20cd680&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4775/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4775?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4775      +/-   ##
==========================================
+ Coverage   77.08%   77.10%   +0.01%     
==========================================
  Files         128      128              
  Lines       21059    21059              
==========================================
+ Hits        16234    16237       +3     
+ Misses       4825     4822       -3     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4775?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4775/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `90.03% <0.00%> (+0.12%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4775/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `88.96% <0.00%> (+0.16%)` | :arrow_up: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4775/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `74.27% <0.00%> (+0.41%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4775?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4775?src=pr&el=footer). Last update [17a88d3...b1ef414](https://codecov.io/gh/huggingface/transformers/pull/4775?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Thanks for sharing, and great dataset. (consider uploading it to [`nlp`](https://github.com/huggingface/nlp)?)

> Thanks for sharing, and great dataset. (consider uploading it to [`nlp`](https://github.com/huggingface/nlp)?)

Thanks for merging. Sure, will do asap !