noamaanMulla-03/transformers-issues huggingface dataset:
[MT5] More docs
Author: patrickvonplaten
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

^^I knew that I forgot something with the docs...

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Comments:
cc @sgugger for notification.
Add output control for TFGPT2LMHeadModel
Author: mymusise
Pull request: yes

# What does this PR do?

Hi, guys. This pull request is going to add a parameter for `GPT2Config` and `TFGPT2LMHeadModel`, to control whether return multi-layer logits or not when training and fine-tuning.

Related issue #8503

Before this change, we need to assign the 'special' loss and metric:
```python
class MyMetrice(tf.keras.metrics.SparseCategoricalAccuracy):

    def update_state(self, y_true, y_pred, sample_weight=None):
        # It will receive output from all layers by default.
        if len(y_pred.shape) > 3:
            return 0
        return super(Mymetrice, self).update_state(y_true, y_pred, sample_weight)

model = TFGPT2LMHeadModel.from_pretrained("distilgpt2")
loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
metric = MyMetrice('accuracy')
model.compile(
    optimizer=optimizer,
    loss=[loss, *[None] * model.config.n_layer],    # It's hard to guess if there is no example
    metrics=[metric]
)
```
After this change, we can train or finetune samples easily.
```python
model = TFGPT2LMHeadModel.from_pretrained("distilgpt2", return_dict=False)
model.compile(
    optimizer=optimizer,
    loss=model.compute_loss,
    metrics=[tf.keras.metrics.SparseCategoricalAccuracy]
)

```

 GPT2: @LysandreJik, @patrickvonplaten
 tensorflow: @jplu

Hope this helps.

Comments:
Hello!

Apparently you are rewriting the  `output_attentions` and `output_hidden_states` parameters.

Thanks for your advice. Yes, so it's a bad idea to only return `last_hidden_state` like this?

I mean, why you are not using the `output_attentions` or `output_hidden_states` parameters that basically do what you are proposing?

I'm sorry to make you confused, maybe my mastery of gpt2 is not enough. Could you please tell me where `output_attentions` and 
`output_hidden_states`(except last) will be used during training and text generation, I can't find the answer from the source code directly. 

I mean, if `output_attentions` or `output_hidden_states` is not using basically, it's better to hide it by default? And call something like `return_multilayer=True` to return those if we need it.

Thanks in advance. :)

No worries! Everything is nicely explained in the [documentation](https://huggingface.co/transformers/model_doc/gpt2.html#tfgpt2lmheadmodel):

```
output_attentions (bool, optional) – Whether or not to return the attentions tensors of all attention layers. See attentions under returned tensors for more detail.
output_hidden_states (bool, optional) – Whether or not to return the hidden states of all layers. See hidden_states under returned tensors for more detail.
```

Thank @jplu , this RP exactly rewriting the `output_attentions` and `output_hidden_states` parameters of `GPT2Config`, close it.



You're welcome, happy to help :)
tokenization_bart.py: return_tensors default should be "pt"
Author: Mehrad0711
Pull request: yes

return_tensors default should be "pt" in bart's `prepare_seq2seq_batch`.

Comments:
Tokenizers are supposed to be framework (PyTorch/TensorFlow/FLAX) agnostic so we probably don't want to do in that direction.

Gotcha. Is this going to be the case for all tokenizers in the future? Because currently they default to PyTorch except for Bart's.
Also, I think the docstring for Bart tokenizer's `return_tensors` needs to be updated then since it says: `optional`, defaults to "pt"

The fact that the BART-like tokenizers have `return_tensors="pt"` is a mistake. The tokenizers should be framework-agnostic.

We will have to update this, which will be a breaking change, so we'll try to put it in the v4.0.0. Do you want to open a PR to fix the issue?

Sure. I'll close this then and make a new PR for that.


Hi @Mehrad0711! We're rushing to `transformers@v4.0.0`, and we don't want that in this release. I've taken the liberty of fixing it in https://github.com/huggingface/transformers/pull/8599. Sorry about that, I hope you had not started your development.

If you have, you can push your fixes and open a PR and I'll incorporate those changes in my PR and mark you as co-author.

No problem @LysandreJik . Thanks for fixing it!
Tokenizers should be framework agnostic
Author: LysandreJik
Pull request: yes
Labels: name: model card

The `prepare_seq2seq_batch` method should not return PyTorch tensors by default. It does not in the base class, and all our tokenizer methods should be agnostic to the framework.

Updated Marian, Pegasus, mBART, FSMT that had `return_tensors="pt"` and RAG that had `return_tensors="np"`.

The documentation for these methods was inconsistent, added the docstrings via the decorator where it was needed.

Comments:
Can't ask you to review here @stas00 but would love your review as FSMT is impacted by this.

LGTM
Add Harry Potter Model Card
Author: ceostroff
Pull request: yes
Labels: name: model card

# What does this PR do?

We made this model that creates new Harry Potter fanfiction based off of popular stories. We hope this will be a fun and useful tool. 

Comments:
That's really cool, thanks for sharing!

[model page](https://huggingface.co/ceostroff/harry-potter-gpt2-fanfiction)
Fixed link to the wrong paper.
Author: cronoik
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

## Before submitting
- [ x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.


 documentation: @sgugger


Comments:
How to train EncoderDecoderModel using bert for seq-to-seq model
Author: jithincheriyan
Pull request: no

Hi @patrickvonplaten,

I am trying to make a sequence to sequence model using EncoderDecodermodel and BERT
Please find the following code,

```python
import pandas as pd
from transformers import EncoderDecoderModel

train_data = [
    ["one", "1"],
    ["two", "2"],
            ]

train_df = pd.DataFrame(train_data, columns=["input_text", "target_text"])

eval_data = [["three", "3"]]
eval_df = pd.DataFrame(eval_data, columns=["input_text", "target_text"])

#using BERT encoder decoder
model=EncoderDecoderModel.from_encoder_decoder_pretrained('bert-base-uncased', 'bert-base-uncased')
model.train(train_df)
results=model.eval([eval_df],)
#results=model.generate([["Five"],])
print(results)
```

But while evaluating, ends up in an error as shown in the figure

![image](https://user-images.githubusercontent.com/52187221/99480246-95474b00-29bc-11eb-88c1-8ffaa88e1738.png)

Any suggestions on the same?



Comments:
Hey @jithincheriyan, 

Please note that `.eval()` and `.train()` are not used to evaluate / train the model in PyTorch. In PyTorch `.eval()` and `.train()` are simply used to set the model into "training" or "evaluation" model. See the functions' API here: https://pytorch.org/docs/master/generated/torch.nn.Flatten.html#torch.nn.Flatten.train

Please refer to this blog post to understand how to train an `EncoderDecoderModel`: https://huggingface.co/blog/warm-starting-encoder-decoder#warm-starting-encoder-decoder-models-with-%F0%9F%A4%97transformers-practice
Missing `tokenizers` file?
Author: guotong1988
Pull request: no

In latest https://github.com/huggingface/transformers/blob/master/src/transformers/tokenization_utils_base.py

![image](https://user-images.githubusercontent.com/4702353/99480139-759b2d00-2992-11eb-8763-87b6b246819a.png)

But there is no `tokenizers` file

Comments:
Hi,

tokenizers is not a file. It's an entire [library](https://github.com/huggingface/tokenizers) built by the Hugging Face team. The code that you show will import some functions from that library, if it's available.
Hosting and online deployment of a transformer chatbot (built with huggingface library)
Author: AlexTS1980
Pull request: no

I'm building a chatbot using BERT for a startup company. At some point it will be deployed online. It turns out, most chatbot hosting services actually want to sell you a chatbot rather than host the one you developed, which is obviously not an option for us, especially that the solution is opensource (pytorch+huggingface). 

I would like to know of a hosting service that 1) accepts custom-built chatbot solution, 2) can accommodate up to 500 online users at any given time, 3) does not charge exorbitant prices. We can't buy a solution, or even augment an existing one, because it is a specific requirement of the project funding (research rather than development). 


Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discusss.huggingface.co) instead?

Thanks!
Improving performance results for BERT 
Author: Stimmot
Pull request: no

I'm using the bert-base-german-cased model to perform token classification with custom NER labels on a dataset of German court documents. I have 11 labels in total (including the O label), which are however not tagged in BIO form. I'm letting the model train and evaluate on an NVidia GeForce GTX Titan X. 

But despite the good ressources and the model, which was actually pretrained on German judicial documents, the results are rather lacking. 

```
precision    recall  f1-score   support

                              Date       0.87      0.99      0.93       407
                   Schadensbetrag       0.77      0.58      0.66       112
                            Delikt       0.59      0.50      0.54        44
                    Gestaendnis_ja       0.60      0.71      0.65        21
                    Vorstrafe_nein       0.00      0.00      0.00         6
Strafe_Gesamtfreiheitsstrafe_Dauer       0.76      0.91      0.83        35
          Strafe_Gesamtsatz_Betrag       0.42      0.52      0.46        25
           Strafe_Gesamtsatz_Dauer       0.52      0.82      0.64        28
                 Strafe_Tatbestand       0.30      0.29      0.30       283

                        micro avg       0.65      0.68      0.66       961
                        macro avg       0.54      0.59      0.56       961
                     weighted avg       0.64      0.68      0.66       961


```

What could be some steps to improve these results? 
Perhaps it's the low data count for some of the labels, or that the labels often are not single tokens but text spans of multiple tokens?

I would be glad for every hint of some more experienced users. I can also share data or other files, if they are relevant. 

This is my config file:

```
{
    "data_dir": "./Data",
    "labels": "./Data/labels.txt",
    "model_name_or_path": "bert-base-german-cased",
    "output_dir": "./Data/Models",
    "task_type": "NER",
    "max_seq_length": 180,
    "num_train_epochs": 6,
    "per_device_train_batch_size": 48,
    "seed": 7,
    "fp16": true,
    "do_train": true,
    "do_predict": true,
    "do_eval": true
}
```

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discusss.huggingface.co) instead?

Thanks!
Fix missing space in unavailable PyTorch/TensorFlow warning
Author: mipo57
Pull request: yes

# What does this PR do?
Fixes missing space in unavailable PyTorch/TensorFlow warning

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSTM: @stas00
 -->


Comments:
tf_bart typo - self.self.activation_dropout
Author: ratthachat
Pull request: yes

# What does this PR do?
Fix one line typo in `modeling_tf_bart` :

`self.self.activation_dropout` -> `self.activation_dropout`

BTW, there's no error in forward pass. 
I met the error when I played around with `model.fit()` :)

## Who can review ?
@sshleifer 

Comments:
LGTM, cc @LysandreJik. I won't merge on my own anymore.
Fixes the training resuming with gradient accumulation
Author: sgugger
Pull request: yes

# What does this PR do?

As #5605 pointed out, there was a mistake in the way the number of steps to skip was computed when the `Trainer` wants to resume training from a checkpoint with gradient accumulation activated. This PR fixes that and adds more tests. More specifically:

1. It tests a regular gradient accumulation training (wasn't done before) and checks it gives the same results as the same training with the batch size multiplied by the number of gradient accumulation steps.
2. It adds a test of a training resuming with gradient accumulation (which fails on current master)
3. It fixes master so that the test in 2 passes.

Fixes #5605

Comments:
Adding PrefixConstrainedLogitsProcessor
Author: nicola-decao
Pull request: yes

# What does this PR do?
This pull request adds a new decoding strategy that constrains the next token to generate based on a callable function. It is a new PR that fixes https://github.com/huggingface/transformers/pull/7784 since the generate function went trough refactoring. @patrickvonplaten 

Comments:
@patrickvonplaten I think we are ready to merge now 😊

Hey @nicola-decao,

Thanks a lot for re-implementing your PR! A couple of things I think we should improve.

1) I'm still not 100% sure how the API of `prefix_allowed_tokens_fn` would look like. Does it take one or two arguments? Also it would be very useful to correctly define the type hints in this case as I mentioned in the comment above

2) Can we add a test for this logits processor? It should be relatively straight-forward in /home/patrick/hugging_face/transformers/tests/test_generation_logits_process.py. The test should only test the `prefix_allowed_tokens_fn` not the whole generate function. 

3) It would be awesome if the docstring could be a bit more explicit (*e.g.* including your paper and maybe even a tiny example)

4) Can we delete the `src/.DS_Store ` file?

5) (Not required for merge) I think we could speed up this function by just using torch.Tensor operations (see comment above), but I'm happy to keep this for a future PR.

Also, let me know if you need help! :-) 

> Great looks good to me now! @yjernite - do you want to take a final look as well?

Yup! Will take a look by EOD

LGTM, thanks for implementing this functionality!

Great work @nicola-decao 

> Great work @nicola-decao

Thanks :)
Cannot train model from scratch using `run_mlm.py`.
Author: GuillemGSubies
Pull request: no

Looks like the trainer does not like when it gets a `None`, so when we train from scratch, there is a `None` in this `if` and crashes:

https://github.com/huggingface/transformers/blob/a6cf9ca00b74a8b2244421a6101b83d8cf43cd6b/examples/language-modeling/run_mlm.py#L357

I solved it by deleting that line, but I guess it could affect to other use cases.


To reproduce, call `run_mlm` this way (I guess it is easier to reproduce, but this might be enough):
```
python  run_mlm.py \
    --model_type bert \
    --train_file ./data/oscar_1000.txt \
    --validation_file ./data/oscar_1000_valid.txt \
    --output_dir testing_model \
    --tokenizer_name bert-base-spanish-wwm-cased  \
    --overwrite_output_dir \
    --do_train \
    --do_eval \
    --evaluation_strategy steps \
    --per_device_train_batch_size 16 \
    --per_device_eval_batch_size 16 \
    --max_steps 500 \
    --save_steps 2000 \
    --save_total_limit 15 \
    --overwrite_cache \
    --max_seq_length 512 \
    --eval_accumulation_steps 10 \
    --logging_steps 1000 \
```

The dataset I'm using I guess that isn't relevant so any corpus will do.
 
 @sgugger

Comments:
Mmm, that is weird as `None` is the default for that argument. Will investigate this when I'm finished with v4 stuff, thanks for flagging!