noamaanMulla-03/transformers-issues huggingface dataset:
Remove comma that turns int into tuple
Author: LysandreJik
Pull request: yes

cf @stefan-it's error on Slack

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4208?src=pr&el=h1) Report
> Merging [#4208](https://codecov.io/gh/huggingface/transformers/pull/4208?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/cafa6a9e29f3e99c67a1028f8ca779d439bc0689&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4208/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4208?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4208      +/-   ##
==========================================
- Coverage   78.32%   78.31%   -0.02%     
==========================================
  Files         120      120              
  Lines       19854    19854              
==========================================
- Hits        15551    15548       -3     
- Misses       4303     4306       +3     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4208?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/trainer.py](https://codecov.io/gh/huggingface/transformers/pull/4208/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmVyLnB5) | `39.88% <ø> (-0.30%)` | :arrow_down: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4208/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.96% <0.00%> (-0.42%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4208/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `90.96% <0.00%> (-0.13%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4208?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4208?src=pr&el=footer). Last update [cafa6a9...bfbed71](https://codecov.io/gh/huggingface/transformers/pull/4208?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Superseded and fixed by #4213
Use with_suffix to change the extension of the path
Author: BramVanroy
Pull request: yes

As per https://github.com/huggingface/transformers/pull/3934#discussion_r421307659. Note that this does not rename the file, it simply changes the path 'string' to another one with a different suffix.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4203?src=pr&el=h1) Report
> Merging [#4203](https://codecov.io/gh/huggingface/transformers/pull/4203?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/026097b9ee7862905ec3f3b0e729c5dbc95a0fd9&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `0.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4203/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4203?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4203      +/-   ##
==========================================
+ Coverage   78.43%   78.44%   +0.01%     
==========================================
  Files         120      120              
  Lines       19800    19797       -3     
==========================================
  Hits        15530    15530              
+ Misses       4270     4267       -3     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4203?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/hf\_argparser.py](https://codecov.io/gh/huggingface/transformers/pull/4203/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9oZl9hcmdwYXJzZXIucHk=) | `61.97% <0.00%> (+1.16%)` | :arrow_up: |
| [src/transformers/trainer.py](https://codecov.io/gh/huggingface/transformers/pull/4203/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmVyLnB5) | `41.83% <0.00%> (-0.35%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4203/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `92.77% <0.00%> (+0.16%)` | :arrow_up: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4203/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `74.38% <0.00%> (+0.41%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4203?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4203?src=pr&el=footer). Last update [026097b...a73d487](https://codecov.io/gh/huggingface/transformers/pull/4203?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Thanks!
Requesting model for TFAlbertForQuestionAnswering
Author: jarednielsen
Pull request: no

# 🌟 New model addition

Is there support for adding a TensorFlow version of the AlbertForQuestionAnswering model? I would be happy to contribute the work. This would also enable the `run_squad.py` example script for TensorFlow.

It also looks like [the preprocessing of SQuAD data is different](https://github.com/huggingface/transformers/blob/master/src/transformers/data/processors/squad.py#L349) for PyTorch and TensorFlow. PyTorch has an option to return `all_example_index`, while TensorFlow does not. This means that running the model evaluation script (which takes the max F1/EM over all answers) is only possible in PT. Running SQuAD evaluation in TensorFlow is important for my use case, and I would love to understand the design decisions or difficulties that went into that decision. Again, happy to contribute - any support or objections for aligning the data preprocessing techniques?


Comments:
Hi! We don't currently have a `run_tf_squad` script, but we would appreciate a contribution! The preprocessing of SQuAD shouldn't be different between pytorch and tensorflow. We haven't gotten to testing that as we haven't gotten to writing that script yet.

If we're to have a TF SQuAD script, we would have to align the pre-processing techniques as well!

And we would definitely welcome a PR introducing `TFAlbertForQuestionAnswering`!

Resolved in https://github.com/huggingface/transformers/commit/6d00033e97e1751a897f2317fdfd35dd853cee29 .
Tpu trainer
Author: LysandreJik
Pull request: yes

This PR aims to bring TPU support to the trainer. It runs on GLUE/MRPC but is yet untested on others.

Left to do:

- [ ] Saving and reloading mid-training
- [ ] Check it runs on a few examples (Language modeling, NER, other GLUE tasks)
- [ ] Write down training speed-ups
- [ ] Write down evaluation speed-ups

The API is not final.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4146?src=pr&el=h1) Report
> Merging [#4146](https://codecov.io/gh/huggingface/transformers/pull/4146?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/d713cfc5ebfb1ed83de1fce55dd7279f9db30672&el=desc) will **decrease** coverage by `0.08%`.
> The diff coverage is `33.33%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4146/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4146?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4146      +/-   ##
==========================================
- Coverage   78.84%   78.76%   -0.09%     
==========================================
  Files         114      114              
  Lines       18688    18729      +41     
==========================================
+ Hits        14735    14751      +16     
- Misses       3953     3978      +25     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4146?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/trainer.py](https://codecov.io/gh/huggingface/transformers/pull/4146/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmVyLnB5) | `42.27% <25.58%> (-1.64%)` | :arrow_down: |
| [src/transformers/training\_args.py](https://codecov.io/gh/huggingface/transformers/pull/4146/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmluZ19hcmdzLnB5) | `84.21% <63.63%> (-3.49%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4146/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `92.77% <0.00%> (+0.16%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4146?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4146?src=pr&el=footer). Last update [d713cfc...e882471](https://codecov.io/gh/huggingface/transformers/pull/4146?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Awesome!
W&B integration improvements
Author: vanpelt
Pull request: yes

Hey guys,

I'm one of the founders of W&B and have been working with @borisdayma on the wandb integration.  I made a few changes to address a few cases we thought through:

1. If wandb is installed but the user hasn't logged in we now print a warning instead of prompting for an api_key
2. Made messaging more clear on how to install wandb and linked to more documentation
3. Added an environment variable for disabling wandb.  This is important for users using fastai or lightning with their own wandb callback.
4. Added an environment variable for disabling wandb.watch.  Because we're pulling gradients and parameters during training there is a small performance impact on training.  We wanted to make it easy for users to disable this feature.

Happy to iterate on any feedback you guys have.  Really excited about this integration!

Comments:
^^ I borked your PR unfortunately (💩). Sorry about that. Care to open a new one?
feat(Trainer): log epoch and final metrics
Author: borisdayma
Pull request: yes

This PR adds new logging capabilities to `Trainer`:

* logs epoch -> this let us compare runs with different batch sizes, using "epoch" as the x-axis instead of "global_step"
* logs final metrics -> metrics are also logged by `Trainer.evaluate` at the end of the script (or at the start if we just use `--do_eval` and not train).

In below example, we see `epoch` has been used for x-axis and `eval_loss` has one extra point corresponding to the end of the script.
![image](https://user-images.githubusercontent.com/715491/81350982-d57e5600-9088-11ea-8ce6-400bda09bea7.png)

Comments:
^^ I borked your PR unfortunately (💩). Sorry about that. Care to open a new one?

(same for #4211)

Sure, no worry!
KeyError with a fine-tuned model
Author: tbabinet
Pull request: no

I've fine-tuned the allenai's scibert to do some NER using the `run_ner.py` script, but I keep bumping into a key error. My code is the following:
```
m = AutoModel.from_pretrained("models/sciBERT_ner_e50/")
tok = AutoTokenizer.from_pretrained("allenai/scibert_scivocab_uncased")
nlp = pipeline("ner", model=m, tokenizer=tok) 
```
where models/scibert_ner_e50 contains my model and the files it needs to run properly.

when i run something like `nlp(abstract)`, the code crashes and sends me the following error:
```
KeyError                                  Traceback (most recent call last)
<ipython-input-86-793f89d86633> in <module>
      3         if(len(row.abstract.split())<512):
      4             abst = row.abstract
----> 5             print(nlp(abst))
      6 
      7     except ValueError as ve:

c:\users\tbabi\appdata\local\programs\python\python37\lib\site-packages\transformers\pipelines.py in __call__(self, *texts, **kwargs)
    792             answer = []
    793             for idx, label_idx in enumerate(labels_idx):
--> 794                 if self.model.config.id2label[label_idx] not in self.ignore_labels:
    795                     answer += [
    796                         {

KeyError: 422
```

Has anyone else been meeting this error ? I've been struggling with this for quiet some time now, and any advice/thoughts/insights on what's going on would be greatly appreciated :) 

Thank you !

Comments:
How many labels/entities are there in your model ? Can you check if the key 422 exists in the models config's id2label by printing it or by opening the config file. You can print config using `model.config` 

Hello
It seems I had simply been misusing my model : I was instantiating an `AutoModel`, when I should have been using the `BertForTokenClassification` class. 
Thanks anyway ! :)
Create README.md
Author: savasy
Pull request: yes
Labels: name: model card


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4132?src=pr&el=h1) Report
> Merging [#4132](https://codecov.io/gh/huggingface/transformers/pull/4132?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/6af3306a1da0322f58861b1fbb62ce5223d97b8a&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4132/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4132?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4132   +/-   ##
=======================================
  Coverage   78.84%   78.84%           
=======================================
  Files         114      114           
  Lines       18689    18689           
=======================================
+ Hits        14735    14736    +1     
+ Misses       3954     3953    -1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4132?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4132/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.55% <0.00%> (+0.41%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4132?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4132?src=pr&el=footer). Last update [6af3306...d6aedbe](https://codecov.io/gh/huggingface/transformers/pull/4132?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Thanks, @savasy! Can you add code fences around the code blocks?
Create README.md
Author: mrm8488
Pull request: yes
Labels: name: model card

model card for my De Novo Drug discovery model using MLM

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4179?src=pr&el=h1) Report
> Merging [#4179](https://codecov.io/gh/huggingface/transformers/pull/4179?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/ff8ed52dd8c6268f2535c0721cde9e360fbb0ce0&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4179/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4179?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4179   +/-   ##
=======================================
  Coverage   78.79%   78.79%           
=======================================
  Files         114      114           
  Lines       18712    18712           
=======================================
  Hits        14745    14745           
  Misses       3967     3967           
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4179?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4179?src=pr&el=footer). Last update [ff8ed52...c11fb93](https://codecov.io/gh/huggingface/transformers/pull/4179?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Really cool. Do they have a downstream task that we can fine-tune on, and get some eval results?

I will check it again and will let you know.
I am also working on a Molecule Drug Transformer for drug target interaction system. It is more powerful and they provide several downstream tasks and evaluation datasets. Something like this: https://arxiv.org/abs/2004.11424
where is the script of a second step of  knwoledge distillation on SQuAD 1.0?
Author: c0derm4n
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

<!-- A clear and concise description of the question. -->
In Distil part, there is a paragraph description which is "distilbert-base-uncased-distilled-squad: A finetuned version of distilbert-base-uncased finetuned using (a second step of) knwoledge distillation on SQuAD 1.0. This model reaches a F1 score of 86.9 on the dev set (for comparison, Bert bert-base-uncased version reaches a 88.5 F1 score)."
so where is the script of "a second step of  knwoledge distillation on SQuAD 1.0" mentioned above?
Thanks a lot, it will be very helpful to me!


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Check here: https://github.com/huggingface/transformers/blob/master/examples/distillation/run_squad_w_distillation.py
[TPU] Doc, fix xla_spawn.py, only preprocess dataset once
Author: julien-c
Pull request: yes

- Fixed the `xla_spawn.py` launcher script which I broke earlier today when moving files around. It should be reasonably more robust now.
- Started to add a little documentation
- Changed the dataset logic so that multiple workers don't concurrently preprocess the dataset.



Comments:
Ok, merging this!
MarianMTModel: Runtime Errors
Author: sshleifer
Pull request: no
Labels: name: marian

11/1010 models cannot me loaded because of state_dict mismatches (not tokenizer).

```python
failing = ['sv-tpi',
 'fi-en',
 'sv-ty',
 'sv-toi',
 'en-st',
 'en-tl',
 'sv-ts',
 'sv-tvl',
 'en-fi',
 'en-ml',
 'fi-sv']
for mname in failing:
	slug = f'opus-mt-{mname}'
	MarianMTModel.from_pretrained(slug)
```

`en-st` is a slightly different error:
```
  'Unable to load vocabulary from file. Please check that the provided vocabulary is accessible and not corrupted.'),
```

Comments:
Fixed on S3.
Adaptive Attention Span for Transformers
Author: djstrong
Pull request: no
Labels: name: wontfix

# 🌟New model addition

## Model description

<!-- Important information -->

## Open Source status

* [x] the model implementation is available: https://github.com/facebookresearch/adaptive-span
* [x] the model weights are available: get_pretrained.sh
* [x] who are the authors: Facebook Research

## Additional context

No additional dependencies required.


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Any clue about how to integrate that into a BERT model?
🌟New model addition: VL-BERT
Author: Eurus-Holmes
Pull request: no
Labels: name: wontfix

# 🌟New model addition

## Model description

[VL-BERT: PRE-TRAINING OF GENERIC VISUALLINGUISTIC REPRESENTATIONS](https://arxiv.org/pdf/1908.08530.pdf)

> *We introduce a new pre-trainable generic representation for visual-linguistic tasks,
called Visual-Linguistic BERT (VL-BERT for short). VL-BERT adopts the simple yet powerful Transformer model as the backbone, and extends it to take both
visual and linguistic embedded features as input. In it, each element of the input is either of a word from the input sentence, or a region-of-interest (RoI) from
the input image. It is designed to fit for most of the visual-linguistic downstream
tasks. To better exploit the generic representation, we pre-train VL-BERT on the
massive-scale Conceptual Captions dataset, together with text-only corpus. Extensive empirical analysis demonstrates that the pre-training procedure can better
align the visual-linguistic clues and benefit the downstream tasks, such as visual
commonsense reasoning, visual question answering and referring expression comprehension. It is worth noting that VL-BERT achieved the first place of single
model on the leaderboard of the VCR benchmark.*

<!-- Important information -->

## Open Source status

* [x] the model implementation is available: https://github.com/jackroos/VL-BERT
* [x] the model weights are available: https://github.com/jackroos/VL-BERT/blob/master/model/pretrained_model/PREPARE_PRETRAINED_MODELS.md

## Additional context

The pre-trainable generic representation for visual-linguistic tasks is becoming more and more important.

<!-- Add any other context about the problem here. -->


Comments:
We have released the code for VL-BERT: https://github.com/jackroos/VL-BERT. Thanks for your attention!

@jackroos Awesome! Thanks for your work!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


@thomwolf @srush Any plans to integrate any visual BERT kind of model (like VilBERT, LXMERT or VL-BERT) in the near future? 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
XLNet multiple sentence modeling token type ids
Author: jyhanna
Pull request: no
Labels: name: Help wanted

name: Discussion

name: wontfix

name: Core: Modeling

XLNet was designed to handle multiple segment modeling (i.e. > 2 sentences) by using relative segment encodings. For sentence-level classification tasks with arbitrary sentence counts, what is the structure of the segment (token type) ids? I’ve found from the documentation that 2-sequence classification is supported by using `create_token_type_ids` but what about more than two segments?

If more than two segments are supported, would it be correct to infer (from examples in the documentation) that a 3-sentence input with `<cls>` after each sentence (`<sep>` token) should have the form:

0, 0, 0, 0, 0, 1, 2, 2, 2, 2, 3, 4, 4, 4, 4, 5.

where 1, 3, 5 are classification token segment ids? Would the transformers XLNet implementation support segment ids of this form?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.