noamaanMulla-03/transformers-issues huggingface dataset:
Store Predictions on CPU in Every Prediction Iteration (Trainer)
Author: gonglinyuan
Pull request: no
Labels: name: wontfix

# 🚀 Feature request

Store Predictions on CPU in Every Prediction Iteration (Trainer)

## Motivation

Currently, in  [`Trainer._prediction_loop`,](https://github.com/huggingface/transformers/blob/33d7506ea10ca92886fd1bb3b5306a1a720c58fe/src/transformers/trainer.py#L785) the predictions (logits) of the model is stored on GPU/TPU in each iteration. After all iterations are finished, they will be concatenated together and sent to CPU. In this way, the GPU/TPU memory usage will increase linearly during prediction. If the test set is very large, there may be insufficient GPU/TPU memory to finish the prediction phase. To save GPU/TPU memory and allow larger-scale inference, the trainer should instead send a batch of prediction to CPU after each iteration.

However, if we do inference on multiple devices, we would need a `distributed_concat` function to aggregate all predictions from all devices. If the predictions are already stored on CPU, we can no longer aggregate them using NCCL. Therefore, this issue is unsolvable unless we introduce another CPU-based distributed communication mechanism. Still, we can at least solve it in a single-GPU scenario.

## Your contribution

If approved, I can write codes to solve this problem when the inference is not running in the distributed mode.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
How to cosume movement-pruning .h5 models in QnA pipeline
Author: pranavpawar3
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

## Details

Hi, I am working on the movement pruning method for QnA models. I have performed all the [given steps](https://github.com/huggingface/transformers/tree/master/examples/movement-pruning) in order to generate the pruned model and then used [this notebook](https://github.com/huggingface/transformers/blob/master/examples/movement-pruning/Saving_PruneBERT.ipynb) to generate the .h5 model files. 

Though I am facing an issue with consuming this model with `QuestionAnsweringPipeline`.

For loading the model; the config file is copied from `BertForQuestionAnswering`, as the pruning repo does not generate any config file.

`BERT_PRUNED_PATH = SERIALIZATION_DIR+'/dbg' +'/BERT_Pruned/'`

`config = BertConfig.from_json_file(BERT_PRUNED_PATH+'config.json')`

*we used squad_sparse.h5 which is renamed to tf_model.h5*

`model_BERT_pruned =
TFBertForQuestionAnswering.from_pretrained(BERT_PRUNED_PATH+'tf_model.h5',config=config)`

**or** 

`model_BERT_pruned =
BertForQuestionAnswering.from_pretrained(BERT_PRUNED_PATH+'tf_model.h5',config=config,from_tf=True)
`

config = 
{
  "architectures": [
    "BertForQuestionAnswering"
  ],
  "attention_probs_dropout_prob": 0.1,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 768,
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "layer_norm_eps": 1e-12,
  "max_position_embeddings": 512,
  "model_type": "bert",
  "num_attention_heads": 12,
  "num_hidden_layers": 12,
  "pad_token_id": 0,
  "type_vocab_size": 2,
  "vocab_size": 30522
}


For pipeline,

`
fill_mask_qna_BERT_pruned = pipeline(
  "question-answering",
model=model_BERT_pruned,
tokenizer=tokenizer,
framework="tf"
)
`

Now, when I test the pipeline on questions and context, I get very random answers, probably because the model is not getting loaded in a proper fashion as it should be. 

@VictorSanh Can you share the instructions on how to load these .h5 format pruned model using Huggingface modules? Or is there any other way to consume the model.

Comments:
Hello,
I've only work with PyTorch for all the experiments for movement pruning (even though I've converted the checkpoints in the hub to their TF version).
The instructions in the notebook show you how you can load a optimized version of the checkpoint (pruning+quantization) which was saved with hdf5 with an .h5 extension. It is not a tensorflow checkpoint. You would have to adapt the steps in the notebook to do it in tensorflow.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
convert_bart script should support mbart through command line.
Author: sshleifer
Pull request: no
Labels: name: wontfix

Desired command:

```convert_mbart_checkpoint.py --checkpoint_dir --pytorch_dump_path```

This should probably also convert the tokenizer.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Feed forward chunking for all pretrained models
Author: Pradhy729
Pull request: no
Labels: name: wontfix

Based on this card: [Feed forward chunking](https://github.com/huggingface/transformers/projects/9#card-39483681)

@patrickvonplaten 

I'd like to help contribute and implement this for all the other models if this is still pending?

Comments:
Any opinions here? I will create a PR if there is interest and would like to get your ideas and suggestions. @patrickvonplaten @sshleifer 

@patrickvonplaten would be the point person and he is on Vacation until August 3.
In the interim, if you want to start working on this go right ahead. Make sure it's actually faster/needed before you start though. I don't really know.

Hey @Pradhy729, 

Yes it would be great to start a PR to add feed forward chunking to other models. Maybe you can start with BERT in your PR and ping us to get Feedback :-) 

A couple of things to consider:

1) You should probably move the config param `config.chunk_size_feed_forward` to the general `configuration_utils.py` file.

2) As @sshleifer said it would be good to benchmark the gains in a very similar way to this Notebook:
https://github.com/patrickvonplaten/notebooks/blob/master/Reformer_2_4.ipynb

3) as said earlier we should start with BERT and `config.chunk_size_feed_forward`.

Awesome! I will start with BERT and share with you for feedback.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Issue with Summary of the tasks - Named Entity Recognition in Docs
Author: Matt-Munns
Pull request: no

- `transformers` version: 3.3.1
- Platform: Darwin-19.6.0-x86_64-i386-64bit
- Python version: 3.7.3
- PyTorch version (GPU?): 1.6.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help

 examples/token-classification: @stefan-it
 documentation: @sgugger

## Information

I am trying to run the Pytorch version of named entity recognition from the "Summary of the tasks" section in the documentation.

## To reproduce

Steps to reproduce the behavior:

I'm running the exact example from the docs, but will attach the code below


```
from transformers import AutoModelForTokenClassification, AutoTokenizer

import torch

model = AutoModelForTokenClassification.from_pretrained("dbmdz/bert-large-cased-finetuned-conll03-english")

tokenizer = AutoTokenizer.from_pretrained("bert-base-cased")

label_list = [
    "O",       # Outside of a named entity
    "B-MISC",  # Beginning of a miscellaneous entity right after another miscellaneous entity
    "I-MISC",  # Miscellaneous entity
    "B-PER",   # Beginning of a person's name right after another person's name
    "I-PER",   # Person's name
    "B-ORG",   # Beginning of an organisation right after another organisation
    "I-ORG",   # Organisation
    "B-LOC",   # Beginning of a location right after another location
    "I-LOC"    # Location
]

sequence = "Hugging Face Inc. is a company based in New York City. Its headquarters are in DUMBO, therefore very" \
           "close to the Manhattan Bridge."

tokens = tokenizer.tokenize(tokenizer.decode(tokenizer.encode(sequence)))

inputs = tokenizer.encode(sequence, return_tensors="pt")

outputs = model(inputs).logits

predictions = torch.argmax(outputs, dim=2)
```

Running this leads to the error: 

```
Traceback (most recent call last):

  File "test.py", line 21, in <module>

    outputs = model(inputs).logits

AttributeError: 'tuple' object has no attribute 'logits'
```

## Expected behavior

I expect this to run successfully and produce predictions for the example sequence. I've run this example before and it succeeded so I'm not sure what's happening differently now. I feel like I'm making a dumb mistake somewhere, but idk. 
Thanks!

Note: 
changing line 21 to 
`outputs = model(inputs)[0]`
seems to lead to the expected output, but this might not be the kind of behavior you all are looking for. 


Comments:
Indeed, this example (and all the others) is missing a `return_dict=True` in the call to `from_pretrained`. Thanks for flagging, the PR mentioned above will fix this.
Cached TF files cannot be loaded with `from_pretrained` without internet connection
Author: amaiya
Pull request: no

# 🐛 Bug

## Information
For TF models in latest version of `transformers v2.11.0`, when the internet is turned off, `from_pretrained` throws an error even when the model has previously been downloaded to cache:

```python
# does not work (wifi/internet turned off)
model = TFBertModel.from_pretrained('bert-base-uncased', local_files_only=True)
```
Error is:
```
OSError: Can't load weights for 'bert-base-uncased'. Make sure that:

- 'bert-base-uncased' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'bert-base-uncased' is the correct path to a directory containing a file named one of tf_model.h5, pytorch_model.bin.

```
The above error occurs regardless of how `local_files_only` is set. This can be problematic in certain settings (e.g., air-gapped networks, firewalls).

By contrast, for PyTorch models with NO internet, `from_pretrained` works perfectly and correctly loads files from cache both when `local_files_only=False` and `local_files_only=True`, which is a  useful parameter recently added in PR #2930 by @BramVanroy and merged by @LysandreJik:

```python
# works and is fast (wifi/internet turned off)
model = BertModel.from_pretrained('bert-base-uncased', local_files_only=True)

# works but is slow due to outgoing connection attempts (wifi/internet turned off)
model = BertModel.from_pretrained('bert-base-uncased', local_files_only=False)
```

## Environment Info

- `transformers` version: 2.11.0
- Platform: Linux-4.15.0-88-generic-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.4.0 (True)
- Tensorflow version (GPU?): 2.1.0 (True)



Comments:
Hi! This issue should have been solved by https://github.com/huggingface/transformers/pull/5116

Hi @LysandreJik:  I just wanted to let you know that this issue is still not resolved as of v3.02.  If you ensure the model files (and vocab, etc.) exist in the cache, and then you turn off wifi/internet, model loading does not work in TensorFlow (but does work in PyTorch):

This **does not** work:
```python
# wifi off and model files exist in cache
rom transformers import *
model = TFAutoModelForSequenceClassification.from_pretrained('distilbert-base-uncased')
# error is: Cannot load weights for distilbert-base-uncased
```

This **does** work:
```python
# wifi off and model files exist in cache
from transformers import *
model = AutoModelForSequenceClassification.from_pretrained('distilbert-base-uncased')
```


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Thanks, @LysandreJik .  I can confirm that PR #6091 resolves this issue (tested by manually applying fix to a local `transformers==3.3.0` installation).
Update README.md
Author: akshayrkg
Pull request: yes

Making transformers readme more robust.

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), 
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? 


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao 
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

Comments:
Create README.md
Author: allenyummy
Pull request: yes
Labels: name: model card

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), 
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? 


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao 
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

Comments:
Create README.md
Author: thedarkzeno
Pull request: yes
Labels: name: model card

<!-- This line specifies which issue to close after the pull request is merged. -->
Fixes #{issue number}


Comments:
@thedarkzeno If you'd like, it'd be awesome if you could add default input texts in portuguese for https://github.com/huggingface/widgets-server/blob/master/DefaultWidget.ts (you can open a PR)

so the inference widget on your model pages is correctly populated
added script for fine-tuning roberta for sentiment analysis task
Author: DhavalTaunk08
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), 
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? 


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao 
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

Comments:
Fix Ray Tune progress_reporter kwarg
Author: krfricke
Pull request: yes

# What does this PR do?

There's a small error in the Ray Tune kwarg parsing: The expected argument name is `progress_reporter`, not `reporter`.

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), 
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? 


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

@sgugger


Comments:
Sorry by the way for not catching these at the same time. This should be all for now though!

Oh I remembered seeing this last week and thinking: This is wrong, I should fix it... but forgot...
Thanks for following through!