noamaanMulla-03/transformers-issues huggingface dataset:
Deepcopy and pickling fails for modeling_outputs
Author: anirudh2290
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 4.0.0
- Python version: 3.8
- PyTorch version (GPU?):   1.6.0
- Using GPU in script?: N/A

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.
 
 albert, bert, GPT2, XLM: @LysandreJik 
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Speed and Memory Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 TextGeneration: @TevenLeScao 
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten @TevenLeScao
 Blenderbot: @patrickvonplaten
 Bart: @patrickvonplaten
 Marian: @patrickvonplaten
 Pegasus: @patrickvonplaten
 mBART: @patrickvonplaten
 T5: @patrickvonplaten
 Longformer/Reformer: @patrickvonplaten
 TransfoXL/XLNet: @TevenLeScao
 RAG: @patrickvonplaten, @lhoestq
 FSMT: @stas00
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->

## To reproduce

Steps to reproduce the behavior:

```
>>> from transformers.modeling_outputs import BaseModelOutput
>>> import torch
>>> import copy
>>> x = BaseModelOutput(last_hidden_state=torch.ones(1,))
>>> z = copy.deepcopy(x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/lib/python3.8/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/opt/lib/python3.8/copy.py", line 263, in _reconstruct
    y = func(*args)
TypeError: __init__() missing 1 required positional argument: 'last_hidden_state'
>>> import pickle
>>> obj = pickle.dumps(x)
>>> obj_loaded = pickle.loads(obj)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __init__() missing 1 required positional argument: 'last_hidden_state'
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
No failures when using deepcopy or pickle.dumps/pickle.loads

Comments:
Pinging @sgugger, the king of model outputs!

That was very quick :) Thank you @sgugger and @LysandreJik !
Add new SQUAD example
Author: sgugger
Pull request: yes

# What does this PR do?

This PR adds a new example for SQUAD (v1 and v2) for simple models (e.g., not the XLNet/XLM more complex version, another example will follow for those) using the datasets library and all the features of the fast tokenizer to simplify considerably the preprocessing and the post-processing.

I've compared the new version to the old one and did not find major differences when:
- fine-tuning a model on SQUAD v1 or v2 with the old and new script
- evaluation an existing model fine-tuned on SQUAD v1 or v2 with the old and new script

The only difference I found was when evaluating an existing model fine-tuned on SQUAD v1 and evaluating it on SQUAD v2. For those, the new script is a bit less good at predicting the null answers (but those models have crappy results on SQUAD v2 anyway, they are just a bit more crappy).

Further plans are:
- add a subclass of Trainer for QA so that the evaluation is done directly with `trainer.evaluate()`
- add a script for XLNet/XLM

Comments:
I'd like the new examples scripts to stay fairly focused on one problem (at the cost of potentially have some duplicate codes) so they're easy to understand (and tweak) by users. We don't support any kind of datasets either (if your QA dataset has fields with names slightly different than SQUAD for instance), users are supposed to adapt the relevant lines in the code to their needs.

So with that in mind, I'd definitely prefer a separate script :-)
Model templates overhaul
Author: LysandreJik
Pull request: yes

This is the first PR to make the model templates better. It improves the templates themselves, as well as the testing tools around them:

- Re-instantiates the tests in the CI, this time as a separate test.
- Respects the library style, and tests it. These tests ensure that the templates have not diverged from the code base, especially due to the `# Copied from ...`.
- Implements a decoder model, with support for cross attentions, to be used in the encoder-decoder framework.
- Implements the same decoder model in TensorFlow
- Implements multiple types of position embeddings, similarly to BERT.
- Tests every new feature.
- Tokenizer separation between slow and fast
- Soft dependency on `cookiecutter`
- Adds easily tweakable integration tests for both PyTorch and TensorFlow

Things left for overhaul 2 & 3:

- General tokenizer improvements, I'm not happy with their current state and it's not tested. I find it surprisingly difficult to have a template for a tokenizer that is general enough, so I'm probably going to try to cover as much possible use-cases as possible
- Encoder-decoder w/ @patrickvonplaten

Things to improve for this overhaul (1):

- Probably speeding up the model templates test. It's running on github actions right now, and it's quite slow (8 minutes) even though the downloads are cached. Possible options are
    - use CircleCI instead
    - Cache the whole environment
    - Probably others, thinking about it
- The test runs on each commit + when the branch is opened. That's unnecessary. 

Comments:
Thank you all for your reviews. Will do the last changes and update from `LMHead` to `CausalLM`.

@jplu the GA machines are indeed less beefy than the CircleCI ones (and there's a reason for that, we pay CircleCI but not GA).

Closing PR and opening it again because the history is messed up.
[Docs] Fix some typos for group beam search
Author: patrickvonplaten
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


Comments:
fixes #8968
Author: cronoik
Pull request: yes

# What does this PR do?

One of the 3.X releases introduce output objects that replaced the previously returned tuples. This PR updates the transformers notebook to reflect that update.


Fixes #8968


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

@patrickvonplaten

Comments:
The failed circleci test is not related to my PR. :)

Hi @cronoik! There's been a mistake done yesterday, and the history of your branch was messed up by mistake (see the file changes, +/-). Do you mind closing this PR and opening another one? No need to do anything on the branch, just closing this one and opening a new one should be enough. Thank you!
fixes #8968
Author: cronoik
Pull request: yes

**This is the same PR as: [link](https://github.com/huggingface/transformers/pull/8991#issue-534604029). I was asked to create a new one due to a merge mistake.**

# What does this PR do? (Text of the previous PR)

One of the 3.X releases introduced output objects that replaced the previously returned tuples. This PR updates the transformers notebook to reflect that update.


Fixes #8968


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

@patrickvonplaten
@LysandreJik

Comments:
02-transformery.ipynb - output from model only strings 'last_hidden_state', 'pooler_output'
Author: tc64
Pull request: no

Running in NVIDIA Docker Container: nvcr.io/nvidia/pytorch:20.11-py3
Pytorch version: 1.8.0a0+17f8c32
transformers version: 4.0.0-rc-1
Python version: 3.6.10 |Anaconda, Inc.| (default, May  8 2020, 02:54:21)  [GCC 7.3.0]

When running through transformers/notebooks/02-transformers.ipynb, I see the following output at this point:

```python
outputs, pooled = model(tokens_pt)
print("Token wise output: {}, Pooled output: {}".format(outputs.shape, pooled.shape))
```
output for this part:
```
AttributeError                            Traceback (most recent call last)
<ipython-input-47-cda4654dfa83> in <module>
     20 #outputs = model_outs[0]
     21 #pooled = model_outs[1]
---> 22 print("Token wise output: {}, Pooled output: {}".format(outputs.shape, pooled.shape))

AttributeError: 'str' object has no attribute 'shape'
```

This is because the values of `outputs` and `pooled` are strings, `"last_hidden_state"` and "pooler_output", respectively. 
However, the following change (comment out original line and replace it with version where `model` output is captured in a single object whose 0 and 1 indices are accessed) produces the desired result:

```
#outputs, pooled = model(tokens_pt)
model_outs = model(tokens_pt)
outputs = model_outs[0]
pooled = model_outs[1]
print("Token wise output: {}, Pooled output: {}".format(outputs.shape, pooled.shape))
```

I am not sure if this is a PyTorch version thing or what, but was hoping to either get some insight or alert you to something coming up when upgrading to the latest pytorch. Thank you.

Comments:
Exactly same issue while using transformers. I'm using Pytorch 1.7. The solution presented solved the issue
Better warning when loading a tokenizer with AutoTokenizer w/o Sneten…
Author: LysandreJik
Pull request: yes

…cePiece

Currently, initializing a `sentencepiece` `AutoTokenizer` without having `sentencepiece` installed results in the following error:

```
AttributeError: 'NoneType' object has no attribute 'from_pretrained'
```
This improves the error message to:
```
 This tokenizer cannot be instantiated. Please make sure you have `sentencepiece` installed in order to use this tokenizer.
```

Fix #8864

Comments:
Thanks!
Flax Masked Language Modeling training example
Author: mfuntowicz
Pull request: yes

Include a training example running with Flax/JAX framework. (cc @avital @marcvanzee)

TODOs:

- [x] Make the collator working with Numpy/JAX array 
- [x] Make sure the training actually works on larger scale
- [x] Make it possible to train from scratch
- [x] Support TPU (`bfloat16`)
- [ ] Support GPU amp (`float16`)
- [x] Improve overall UX

Comments:
Create README.md
Author: hailabpucpr
Pull request: yes
Labels: name: model card

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


Comments:
[Bart] Cannot use Bart decoder cache with torchscript
Author: patrickvonplaten
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.0.2
- Platform: Linux-4.15.0-111-generic-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.5
- PyTorch version (GPU?): 1.6.0+cpu (False)
- Tensorflow version (GPU?): 2.2.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help

 Bart: @sshleifer

## Information

Model I am using (Bert, XLNet ...): Bart

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

When trying to use torchscript for Bart while passing `decoder_input_ids`:

```python
from transformers import BartModel
import torch

model = BartModel.from_pretrained("sshleifer/bart-tiny-random")
input_ids = decoder_input_ids = torch.tensor([19 * [1] + [model.config.eos_token_id]])
traced_model = torch.jit.trace(model, (input_ids, decoder_input_ids))
```

the following error occurs:
```
RuntimeError: Only tensors, lists, tuples of tensors, or dictionary of tensors can be output from traced functions
```

On the other hand if one disables the past via `model.config.use_cache = False`, then no 
error occurs. This could mean that the cache data structure should be updated to correctly work with Torchscript.

## Expected behavior

No error should occur when using Bart + Torchscript  in the way explained above.


Comments:
After having fixed the bug it would be great if this line: https://github.com/huggingface/transformers/blob/ac001c48b8df1f5aadcc8cf2c71d7c1116c05250/tests/test_modeling_common.py#L252 can be removed so that a test for Bart + `past_key_value` is enabled.

thanks for writing such a good issue, I'll take a look tomorrow.

This should be looked at again after https://github.com/huggingface/transformers/pull/7474 is merged

Refactor resolves the problem -> should be fine after merge
Bart: Instatiate lm_head once without wasting memory
Author: sshleifer
Pull request: no
Labels: name: wontfix

name: seq2seq

Marking this down since I agreed to do it https://github.com/huggingface/transformers/pull/4803#discussion_r443381438

cc @patrickvonplaten 


Comments:
this will happen as part of TPU issue.

Can you link the issue? Is there an open PR for this already? 

This one: https://github.com/huggingface/transformers/pull/5960 , but it's broken afaict.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.