noamaanMulla-03/transformers-issues huggingface dataset:
CL util to convert models to fp16 before upload
Author: sshleifer
Pull request: yes

This should probably go to `commands/`

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5953?src=pr&el=h1) Report
> Merging [#5953](https://codecov.io/gh/huggingface/transformers/pull/5953?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/d1d15d6f2de9e2cde48ff3ea2072add3311ce2ac&el=desc) will **increase** coverage by `0.03%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5953/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5953?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5953      +/-   ##
==========================================
+ Coverage   78.52%   78.56%   +0.03%     
==========================================
  Files         146      146              
  Lines       26314    26314              
==========================================
+ Hits        20664    20674      +10     
+ Misses       5650     5640      -10     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5953?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5953/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `82.48% <0.00%> (+0.28%)` | :arrow_up: |
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5953/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `86.21% <0.00%> (+2.25%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5953?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5953?src=pr&el=footer). Last update [d1d15d6...055e604](https://codecov.io/gh/huggingface/transformers/pull/5953?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
[{m}bart] Fix final_logits bias warning
Author: sshleifer
Pull request: no

This warning is safe to ignore, but can be easily fixed.

```bash
Some weights of BartForConditionalGeneration were not initialized from the model checkpoint at facebook/mbart-large-en-ro and are newly initialized: ['final_logits_bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
```

Comments:
also
```bash
Some weights of the model checkpoint at facebook/mbart-large-en-ro were not used when initializing BartForConditionalGeneration: ['lm_head.weight']
```

Hey @sshleifer 
Want me to pick this one up?

It's non trivial. You probably have to update S3. I would pick another.

Maybe try to understand why 
```
tests/test_pipelines.py::MonoColumnInputTestCase::test_torch_fill_mask_results
```
has started failing, half of #5265
Create README.md
Author: ramsrigouthamg
Pull request: yes
Labels: name: model card

Adding model card - readme

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6032?src=pr&el=h1) Report
> Merging [#6032](https://codecov.io/gh/huggingface/transformers/pull/6032?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/c69ea5efc4eac65b183e8d07b1bf91d20bbe0c8c&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/6032/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/6032?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #6032      +/-   ##
==========================================
+ Coverage   78.50%   78.51%   +0.01%     
==========================================
  Files         146      146              
  Lines       26249    26249              
==========================================
+ Hits        20606    20610       +4     
+ Misses       5643     5639       -4     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/6032?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6032/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `86.46% <0.00%> (+1.00%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/6032?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6032?src=pr&el=footer). Last update [c69ea5e...cbb3748](https://codecov.io/gh/huggingface/transformers/pull/6032?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


@sgugger ! Hi Sylvain, Wanted to get the pull request for this model card approved. If you can take a look that would be great :) Thanks for your time.

Sincerely,
Ramsri
Using the default trainer args
Author: williamFalcon
Pull request: no
Labels: name: wontfix

Hey guys!

Just looked through the transformer_base... why not also allow any of the trainer args to be used?

I imagine the best set up for you guys is:
- add default trainer args (since people can look into the lightning docs for details on this)
- add hf specific args (nlp related or whatever you guys need)
- add model specific args

But instead it looks like you only use a subset of the args
https://github.com/huggingface/transformers/blob/dd9d483d03962fea127f59661f3ae6156e7a91d2/examples/transformer_base.py#L275


Something like this should work:
```python
parser = ArgumentParser()

# enable all trainer args
parser = Trainer.add_argparse_args(parser)

# add the HF args
parser.add_argument(--some_hf_specific_thing, ...)

# add the model args
parser = GoodGAN.add_model_specific_args(parser)

# cook them all up :)
args = parser.parse_args()
```

Check here for mode details:
https://pytorch-lightning.readthedocs.io/en/stable/hyperparameters.html#multiple-lightning-modules

@srush @nateraw 

Comments:
This sounds good. We originally were trying to reimplement exactly the other examples. But now we can have the lightning examples support a wider range of options.

@srush I can take care of this when I go to work on #4494 if that's alright with you.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
DMOZ - web page classification / multi-language
Author: ghost
Pull request: no
Labels: name: wontfix

Hi,

Hope you are all well !

Still quite a newbie with transformers, I wanted to know how could it be possible to build a web page classifier with the DMOZ dump and to classify them into categories in several languages.

Thanks in advance for any insights or inputs on that question.

Cheers,
X

Comments:
Can you check the Dmoz[Dmoz] 
the database in sql he cost 1000$ to change it  (https://idmoz.org) 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
[pack_dataset] don't sort before packing, only pack train
Author: sshleifer
Pull request: yes

this is better than sorting.
But for best metrics, don't pack.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5954?src=pr&el=h1) Report
> Merging [#5954](https://codecov.io/gh/huggingface/transformers/pull/5954?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/d1d15d6f2de9e2cde48ff3ea2072add3311ce2ac&el=desc) will **increase** coverage by `0.03%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5954/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5954?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5954      +/-   ##
==========================================
+ Coverage   78.52%   78.56%   +0.03%     
==========================================
  Files         146      146              
  Lines       26314    26314              
==========================================
+ Hits        20664    20674      +10     
+ Misses       5650     5640      -10     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5954?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5954/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `82.48% <0.00%> (+0.28%)` | :arrow_up: |
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5954/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `86.21% <0.00%> (+2.25%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5954?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5954?src=pr&el=footer). Last update [d1d15d6...97c473a](https://codecov.io/gh/huggingface/transformers/pull/5954?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
slow from_pretrained failures
Author: sshleifer
Pull request: no

github actions has [full traceback](https://github.com/huggingface/transformers/runs/910644181?check_suite_focus=true)

Failures are all like
```python
        for model_name in BERT_PRETRAINED_MODEL_ARCHIVE_LIST[:1]:
            config = BertConfig.from_pretrained(model_name)
            self.assertIsNotNone(config)
            self.assertIsInstance(config, PretrainedConfig)
    
            model = BertModel.from_pretrained(model_name)
            model, loading_info = BertModel.from_pretrained(model_name, output_loading_info=True)
            self.assertIsNotNone(model)
            self.assertIsInstance(model, PreTrainedModel)
            for value in loading_info.values():
>               self.assertEqual(len(value), 0)
E               AssertionError: 1 != 0
```

@sgugger is that from your change?


Comments:
I didn't add any ignore missing key flag to BERT, so that's weird.

Cool. I'll take this one.

It's `position_ids`, from Morgan's change. Will fix.
```
WARNING  transformers.modeling_utils:modeling_utils.py:885 Some weights of BertModel were not initialized from the model checkpoint at bert-base-uncased and are newly initialized: ['bert.embeddings.position_ids']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
WARNING  transformers.modeling_utils:modeling_utils.py:885 Some weights of BertModel were not initialized from the model checkpoint at bert-base-uncased and are newly initialized: ['bert.embeddings.position_ids']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
```
[fix] no warning for position_ids buffer
Author: sshleifer
Pull request: yes

Fixes [#6044](https://github.com/huggingface/transformers/issues/6044)

The failing tests check that there are no missing keys for bart, but morgan's recent change started registering a position_ids buffer in `__init__` for four models, with more expected.

Since we do not expect weights to be saved for position_ids, this PR adds the pattern `position_ids` to authorized_missing_keys for all PT models.


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6063?src=pr&el=h1) Report
> Merging [#6063](https://codecov.io/gh/huggingface/transformers/pull/6063?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/c8bdf7f4ecd73680cb0751d9efc8fa3a992c2c2d&el=desc) will **increase** coverage by `1.19%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/6063/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/6063?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #6063      +/-   ##
==========================================
+ Coverage   77.39%   78.58%   +1.19%     
==========================================
  Files         146      146              
  Lines       26314    26314              
==========================================
+ Hits        20366    20680     +314     
+ Misses       5948     5634     -314     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/6063?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6063/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `87.32% <100.00%> (ø)` | |
| [src/transformers/modeling\_tf\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/6063/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9yb2JlcnRhLnB5) | `43.98% <0.00%> (-49.38%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_mobilebert.py](https://codecov.io/gh/huggingface/transformers/pull/6063/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9tb2JpbGViZXJ0LnB5) | `96.77% <0.00%> (+73.38%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/6063?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6063?src=pr&el=footer). Last update [c8bdf7f...5de946f](https://codecov.io/gh/huggingface/transformers/pull/6063?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


gunna merge this so that github actions catches it.
WIP: haiku bert implementation
Author: madisonmay
Pull request: yes
Labels: name: wontfix

Still a work in progress but the contextual embeddings line up with the pytorch version so this is roughly at parity with jax-bert

TODO (mostly notes to myself): 
- [x] Add `save_pretrained`
- [ ] Make `from_pretrained` work with names
- [ ] Add dropout at training time, pass through training flag
- [ ] Make sure weight initializations line up when pre-trained state isn't passed
- [ ] Gradually work towards parity with the pytorch version if desired? (target models, BERT variants, etc.) 
- [ ] Write HaikuPretrainedModel to take advantage of archive resolution / make saving + loading compatible with pytorch bins?

To use the pre-trained weights cleanly I ended up subclassing `hk.Module` -- unsure how I feel about this decision but I couldn't think of a better method at the time.  Feel free to suggest an alternative if you have ideas.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Pipelines should use tuples instead of namedtuples
Author: LysandreJik
Pull request: yes

Fix https://github.com/huggingface/transformers/issues/5713

The ONNX conversion cannot handle other objects than tuples, lists and variables. Since calling the conversion script via the command line uses a pipeline, and pipelines cannot be configured with specific model kwargs, the change is so that pipelines manage tuples instead of namedtuples (not a breaking change).

Comments:
help：OSError: Model name 'ctrl' was not found in tokenizers model name list (ctrl). We assumed 'ctrl' was a path, a model identifier, or url to a directory containing vocabulary files named ['vocab.json', 'merges.txt'] but couldn't find such vocabulary files at this path or url.
Author: Heiheiyo
Pull request: no

# ❓ Questions & Help
![image](https://user-images.githubusercontent.com/15426714/87268409-dd30ff80-c4fc-11ea-9e34-4d1c06dc1d23.png)


Comments:
As for me, I'm getting 
```
OSError: Can't load weights for 'mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es'. Make sure that:

- 'mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es' is the correct path to a directory containing a file named one of tf_model.h5, pytorch_model.bin.
```

the line of code is 
```
nlp = pipeline(
    'question-answering', 
    model='mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es',
    tokenizer=(
        'mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es',  
        {"use_fast": False}
    )
)
```
but my question is, why does pipeline download the model but can't load/find the weights?? I'm using python:3.7 dockerimage.
I'm using tf version '2.2.0'

@Heiheiyo, is it possible you have a `ctrl` folder that does not contain the vocab and merges files?
When running your command on master I have no issues with CTRL.

@Kreijstal, on what version of transformers are you running? I copy-pasted your command and it works fine on the `master` branch.

@LysandreJik Thank you for your answer.I have solved this problem.I downloaded the ctrl model and modified the model file path.


@LysandreJik I solved this problem too, I used the dockerfiles I found on this repo to figure out the right libraries that might not have been installed
AutoModel.from_pretrained with torchscript flag raises a TypeError: __init__() got an unexpected keyword argument 'torchscript'
Author: jonsnowseven
Pull request: no

# 🐛 Bug

## Information

Model I am using: BertModel and AutoModel

Language I am using the model on: English

## To reproduce

Steps to reproduce the behavior:

```python
from transformers.modeling_auto import AutoModel
from transformers.modeling_bert import BertModel

bert_model = BertModel.from_pretrained('bert-base-uncased', torchscript=True)
bert_model = AutoModel.from_pretrained('bert-base-uncased', torchscript=True)
```
<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Behaviour

`bert_model = AutoModel.from_pretrained('bert-base-uncased', torchscript=True)` raises a 
`TypeError: __init__() got an unexpected keyword argument 'torchscript'`

## Expected behaviour

Successfully create a BertModel object using AutoModel class.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 2.9.0
- Platform: `Darwin Kernel Version 19.4.0: Wed Mar  4 22:28:40 PST 2020; root:xnu-6153.101.6~15/RELEASE_X86_64`
- Python version: 3.6.10
- PyTorch version: 1.3.1
- Tensorflow version: Not applicable
- Using GPU in script?: Not applicable
- Using distributed or parallel set-up in script?: Not applicable


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This issue still exists and is relevant to me. Can we get an official response?

Yes, I would also like an official answer, please.

It appears that `AutoConfig` accepts a `torchscript` keyword parameter. The `AutoConfig` object can then be passed as the `config` keyword parameter to `AutoModel`. Hope this workaround helps @jonsnowseven 

Hi! This was fixed by https://github.com/huggingface/transformers/pull/5665. Could you try to install from source and try again?
New Community NB Add
Author: lordtt13
Pull request: yes

Signed-off-by: lordtt13 <thakurtanmay72@yahoo.com>

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5824?src=pr&el=h1) Report
> Merging [#5824](https://codecov.io/gh/huggingface/transformers/pull/5824?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/283500ff9f9041fc027a2ce54b4e1f6337c5abbb&el=desc) will **decrease** coverage by `0.43%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5824/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5824?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5824      +/-   ##
==========================================
- Coverage   78.50%   78.07%   -0.44%     
==========================================
  Files         146      146              
  Lines       26047    26047              
==========================================
- Hits        20448    20335     -113     
- Misses       5599     5712     +113     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5824?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/5824/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9vcGVuYWkucHk=) | `20.27% <0.00%> (-74.92%)` | :arrow_down: |
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5824/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `83.70% <0.00%> (-2.51%)` | :arrow_down: |
| [src/transformers/modeling\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/5824/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19vcGVuYWkucHk=) | `81.02% <0.00%> (-1.29%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/5824/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9yb2JlcnRhLnB5) | `93.36% <0.00%> (+49.37%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5824?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5824?src=pr&el=footer). Last update [283500f...811bc81](https://codecov.io/gh/huggingface/transformers/pull/5824?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).