noamaanMulla-03/transformers-issues huggingface dataset:
Added integration tests for Pytorch implementation of the FlauBert model
Author: spatil6
Pull request: yes

Added integration tests for Pytorch implementation of the FlauBert model


Fixes #9950 


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

@LysandreJik 

Comments:
@LysandreJik Need your help here. Not sure, why test cases are failing.

opening new PR.
AttributeError: module 'transformers' has no attribute 'PegasusForCausalLM'
Author: ibrahim-elsawy
Pull request: no

## Environment info

- `transformers` version: 4.2.2
- Platform: Linux-5.8.0-40-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.7.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help
@sgugger @patil-suraj 
Models:
-tuner007/pegasus_paraphrase




## Information

trying to import "PegasusForCausalLM" from transformers and I got AttributeError: module 'transformers' has no attribute 'PegasusForCausalLM'

## To reproduce

```python

from transformers import PegasusForCausalLM, AutoConfig
model_name = "tuner007/pegasus_paraphrase"
output_logits = True
output_hidden_states = True
p_config =AutoConfig.from_pretrained(
            model_name,
            output_hidden_states=output_hidden_states,
            output_logits=output_logits)
pegasus = PegasusForCausalLM.from_pretrained(model_name, config=p_config)


```
```
AttributeError: module transformers has no attribute PegasusForCausalLM
```



Comments:
Yes, this model was only added recently, it's not in v4.2.1 (check the [documentation](https://huggingface.co/transformers/model_doc/pegasus.html)). 
You need to install the pre-release:
```
pip install transformers --pre
```
or [install from source](https://huggingface.co/transformers/installation.html#installing-from-source) to use it.
Cannot import DataCollatorForSeq2Seq from Transformers library
Author: Arman-IMRSV
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.2.2
- Platform: Ubuntu
- Python version: 3.7
- PyTorch version (GPU?):
- Tensorflow version (GPU?): 1.7.1 - GPU : T4
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?:

### Who can help
@patil-suraj 
@sgugger 


## Information

Model I am using (Bert, XLNet ...): mT5

The problem arises when using:
* [ ] the official example scripts: run_seq2seq.py


The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: Translation
* [ ] my own task or dataset: My own json dataset

## To reproduce

Steps to reproduce the behavior:

```
python run_seq2seq.py \
    --model_name_or_path google/mt5-small \
    --do_train \
    --do_eval \
    --task translation_en_to_fa \
    --train_file Persian/seq2seq_train.json \
    --validation_file Persian/seq2seq_val.json \
    --output_dir translation_task_output \
    --overwrite_output_dir \
    --per_device_train_batch_size=4 \
    --per_device_eval_batch_size=4 \
    --predict_with_generate \
    --text_column text_column_name \
    --max_source_length 64 \
    --max_target_length 64 \
    --max_train_samples 10240 \
    --max_val_samples 512 \
    --source_lang en \
    --target_lang fa \
    --eval_beams 1 \
    --source_prefix "translate English to Persian: "
```

## Error:
```
File "run_seq2seq.py", line 31, in <module>
    from transformers import (
ImportError: cannot import name 'DataCollatorForSeq2Seq' from 'transformers' (unknown location)

```


Comments:
Could you try running this on master ? I just ran this script on master and it's working fine.

As mentioned in the README of the [examples folder](https://github.com/huggingface/transformers/tree/master/examples#important-note), all examples require a [source install](https://huggingface.co/transformers/installation.html#installing-from-source).

`DataCollatorForSeq2Seq` is not in the last release, it was introduced since then.

@sgugger @patil-suraj Thanks :)
[examples/seq2seq] support label smoothing
Author: patil-suraj
Pull request: yes

# What does this PR do?

Add support for label smoothing by adding `prepare_decoder_input_ids_from_labels` method to all seq2seq models which will let us prepare `decoder_input_ids` outside the model. 

For context, we need to pass  `decoder_input_ids`  for label smoothing because we don't pass `labels`  to avoid calculating loss twice, which leads to speeds degradation, see #9713.

@sgugger , @patrickvonplaten  what do we think about adding `prepare_decoder_input_ids_from_labels` to every seq2seq model, there are already `shift_tokens_right/_shift_right` methods, but the name is a bit confusing IMO to use outside the model.

Comments:
> I don't know if the shift methods are used for something else in the seq2seq methods, but if this was their only use, we could maybe deprecate them?

those are used for exactly the same reason, `prepare decoder_input_ids` by shifting `labels`, and those are mostly used inside the models, so yeah, think we could deprecate them

I agree we could remove the `pad_token_id` argument.
[examples] make run scripts executable
Author: stas00
Pull request: yes

For consistently and convenience of not needing to type `python` to run a script this PR adds to all `examples/*/run_*py` scripts a python shebang and make the scripts executable.

Was done with:
```
perl -0777 -pi -e 's|^|#!/usr/bin/env python\n|' */run_*.py
chmod a+x */run_*.py
```

@patil-suraj , @sgugger 

Comments:
Totally!

Just this one:
```
/templates/adding_a_new_example_script/{{cookiecutter.directory_name}}/run_{{cookiecutter.example_shortcut}}.py
```
correct?

Any other scripts that I didn't catch with `examples/*/run_*.py`?


I don't think so, we should be good.
Override Default Params on QnA Pipeline
Author: mabu-dev
Pull request: no

How should we override the default params of QnA Model in Pipelines?

I tried the below but it throws an error?!

![image](https://user-images.githubusercontent.com/22409996/107048748-f586a580-67d1-11eb-88fe-f78901d99734.png)

Comments:
I think you first need to initialize the pipeline, and then call it with any parameters you like. Example:

```
from transformers import pipeline
nlp = pipeline("question-answering")
context = r"""
Extractive Question Answering is the task of extracting an answer from a text given a question. An example of a
question answering dataset is the SQuAD dataset, which is entirely based on that task. If you would like to fine-tune
a model on a SQuAD task, you may leverage the examples/question-answering/run_squad.py script.

result = nlp(question="What is extractive question answering?", context=context, max_answer_len=100)
```

For the documentation of the `__call__` method of `QuestionAnsweringPipeline`, see [here](https://huggingface.co/transformers/main_classes/pipelines.html#transformers.QuestionAnsweringPipeline.__call__).

This solved my issue. Thank you very much.
BertForPreTraining with NSP
Author: choidongyeon
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

## Details
I am trying to train BERT from scratch following a modification of https://huggingface.co/blog/how-to-train, where I use a BertTokenizer and BertForPreTraining. The [documentation for BertForPreTraining](https://huggingface.co/transformers/model_doc/bert.html#transformers.BertForPreTraining) states that it has two heads on top for both pre-training processes (MLM and NSP), but [the example provided](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bert.py#L874-L884) only provides an example of MLM.

Based on [a comment](https://github.com/huggingface/transformers/issues/2693#issuecomment-580870278) provided by @LysandreJik in a previous issue, it seems that none of the provided datasets (i.e. LineByLineTextDataset) will handle the NSP objective and this objective is excluded because the RoBERTa paper has proven that the NSP objective was not particularly helpful.

@LysandreJik additionally noted that anyone who wants to implement the NSP objective can do so by changing the dataset/training loop, and I was wondering if there were any plans to add support for NSP for the sake of completeness? 

It seems that something similar to what is going on in a PR (https://github.com/huggingface/transformers/pull/6168) for Albert SOP can be done. Is this correct and can anyone provide me with some guidance moving forward?

Comments:
Hi! Supporting the NSP objective is not on our roadmap, due to the reason you've linked and because of insufficient bandwidth. 

However, similar to the work in #6168 for SOP, we're very open to contributions and would accept a PR adding the BERT NSP objective to the datacollators/datasets.

Awesome, I've been working on something similar. Will open a PR, thanks!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


@choidongyeon  May i ask if the work on dataset part using in BertForPreTraining APIs is finished?  Any example codes like run_mlm.py (is there a run_mlm_nsp.py?) can help, looking forward to your reply, thx!
BERT adapted to time series
Author: jbechara
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

<!-- A clear and concise description of the question. -->
Is there a better way of modifying BERT to take time series as input (i.e. numerical data instead of text) than editing my local library to skip the word embedding? If not, what is the easiest way to do the latter?

Thanks!

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Can this issue be opened again? I recon there is a need to discuss this possibility

@jbechara / @MJimitater : Hello! 

I happened to stumbled upon this issue earlier this week. We have a paper (with code and a new dataset), which is to appear in ICASSP '21, where we propose to model multivariate times series dataset through BERT and GPT2. Please give it a try to see if it serves your purpose! 

Paper: https://arxiv.org/abs/2011.01843
Code: https://github.com/IBM/TabFormer
[s2s examples] README.md fixes
Author: stas00
Pull request: yes

This PR:
* fixes a cl arg typo
* clarifies that it's jsonlines format and not json that's expected
* adds a link explaining jsonlines

@sgugger, how do we apply the auto-re-wrapping to examples README.md files? Currently the new file is all very long lines. Thank you!

@patil-suraj, @sgugger  

Comments:
seq2seq: fail gracefully when predicting using --device cpu and --fp16
Author: PeterAJansen
Pull request: no

When using the recommended seq2seq evaluation procedure in https://github.com/huggingface/transformers/blob/master/examples/seq2seq/README.md :

```
export DATA_DIR=wmt_en_ro
./run_eval.py t5-base \
    $DATA_DIR/val.source t5_val_generations.txt \
    --reference_path $DATA_DIR/val.target \
    --score_path enro_bleu.json \
    --task translation_en_to_ro \
    --n_obs 100 \
    --device cuda \
    --fp16 \
    --bs 32
```

If `--device cuda` is switched to `--device cpu` then it eventually fails with a pytorch error if `--fp16` is also enabled:

```
"threshold_cpu" not implemented for 'Half'
```

It seems that fp16 and cpu evaluation may currently be incompatible.


Comments:
Added integration tests for Pytorch implementation of the FlauBert model
Author: spatil6
Pull request: yes

Added integration tests for Pytorch implementation of the FlauBert model


Fixes #9950 


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

@LysandreJik 

Comments:
[Good first issue] FlauBERT PyTorch Integration tests
Author: LysandreJik
Pull request: no
Labels: name: PyTorch

name: Tests

name: Good First Issue

The PyTorch implementation of the FlauBERT model currently has no integration tests. This is problematic as the behavior can diverge without being noticed.

The [test_modeling_flaubert.py](https://github.com/huggingface/transformers/blob/master/tests/test_modeling_flaubert.py) file should be updated to include integration testing.

An example of a good modeling integration test is visible in the [test_modeling_bert.py#L552-L565](https://github.com/huggingface/transformers/blob/master/tests/test_modeling_bert.py#L552-L565) file:

https://github.com/huggingface/transformers/blob/1809de5165804666ba6c6a02a9d177f6683869cc/tests/test_modeling_bert.py#L552-L565

Some additional tips:
- The test must be marked as slow using the `@slow` decorator, so as to be run *daily*, and not on every commit of every branch/pull request of this repository.
- The test must be decorated with the `@require_torch` decorator so as to only be run in environments using PyTorch.
- A single test is necessary. If you feel like implementing multiple of these, then sharing the same checkpoint would be ideal so as to reduce download time.
- The TensorFlow implementation already has an integration test, which is visible here:

https://github.com/huggingface/transformers/blob/1809de5165804666ba6c6a02a9d177f6683869cc/tests/test_modeling_tf_flaubert.py#L342-L370

This test can be translated to PyTorch.

Comments:
Check copies match full class/function names
Author: sgugger
Pull request: yes

# What does this PR do?

@LysandreJik pointed out a failure in the `check_copies` script that came from the fact that when looking for the code of an object, the script was matching the first line it found that begun with the name of the object. So it would match `DebertaLayer` with `DebertaLayerNorm` and compare those.

This PR fixes that.

Comments:
Do not allow fine tuning with sequence size larger than during training
Author: ioana-blue
Pull request: no

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

## Motivation

I just wasted some hours chasing a problem see #10010  that I think could be avoided with the following simple solution: 

Do not allow `max_seq_length` to be higher than `max_position_embeddings`. 

Most models are built with 512 and, as such, this problem doesn't happen too often. It so happens that BERTweet was trained with 130. The code allows it to run with `max_seq_length` higher than 130 and it ends up with cryptic cuda errors down the pipe. 




Comments:
Also, I for one, I find it confusing that there are two different parameters that (kind of) refer to the same thing. I was lucky that I had run my code successfully with 10+ other models from the hub and it stood out that the sizes used for embeddings during training were different.
Restore TF embeddings and attention layers to their previous version
Author: jplu
Pull request: yes

# What does this PR do?

This PR restores the attention layers and the embeddings as it was in v4.1, even though the embeddings got few improvements compared to their original version to keep XLA compliancy. The reason is because we realized that some used operators were not compatible with some NN SDKs such as the one from Qualcomm or ONNX.


Comments:
Pinging @mfuntowicz 

> Morgan was mentioning that the transpose_for_score method was called right after the Q/K/V projection, but that there was no need to split this dimension if we're not doing head masking.
What do you think? Maybe that's some work for another PR, though.

I think it seems doable but not sure, I prefer to keep things like this to be sure we revert properly as it was before and we get at least a proper version, and we can take care of this change in another PR.

Sounds good to me!

LGTM! @patrickvonplaten feel free to merge if you approve the changes ^^

I won't have time to do a proper review today (can do it tomorrow), but feel free to merge without me if @LysandreJik and @sgugger are ok with it

@patrickvonplaten if you can take a look at it today and merge it if it's fine with you, that would be great
Fix mlflow parameter overflow
Author: noise-field
Pull request: yes

# What does this PR do?

This PR fixes the issue #8849 where MLflow logging failed due to parameters logged being too long. Now the MLflow logger also fetches the limits directly from MLflow validation utility.

<!-- Remove if not applicable -->

Fixes #8849


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@sgugger 



Comments:
Thanks!

Sorry again we let this sit for so long!

So since it's been a long time, the diff has gotten quite messy. Would you mind closing and re-opening a clean PR @noise-field ? Ping me on it and we'll expedite the review. Sorry again.

Closing as requested
Fix TF template
Author: jplu
Pull request: yes

# What does this PR do?

Fix the TF template.

Comments:
@LysandreJik I think there is a problem with the Template test. It doesn't seem to take into account the changes in the current PR.

Good catch @jplu, thanks for fixing! Yes indeed, this test needs to be reworked. Tracking it in https://github.com/huggingface/transformers/issues/10065
Added integration tests for Pytorch implementation of the ELECTRA model
Author: spatil6
Pull request: yes

Added integration tests for Pytorch implementation of the ELECTRA model


Fixes #9949 


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

@LysandreJik 

Comments:
@LysandreJik  Need your help here. Not sure, why test cases are failing.

closing this PR, due to git conflict.
BertGenerationTokenizer provides an unexpected value for BertGenerationModel
Author: sadakmed
Pull request: no

- `transformers` version: 4.2.2
- PyTorch version (GPU?): 1.7.0+cu101
- tokenizers: @n1t0, @LysandreJik

## Information

in both models BertGenerationEncoder,  BertGenerationDecoder, there's no need for `token_type_ids` however the BertGenerationTokenizer provides it, this issue will be raised if you want to input the tokenizer results directly with `**`,

and if it meant to be like this, and the user should be aware of this behaviour, I think a change should be in the documentation.

Note: Another issue with BertGenerationTokenizer is the necessity of sentencepiece module, do you prefer that it should for the user to install it separately or it should be included in transformers dependencies.


Comments:
Hi @sadakmed!

You're right, there's no need for token type IDs in this tokenizer. The workaround for this is to remove `token_type_ids` from the model input names, as it is done in the DistilBERT tokenizer:

https://github.com/huggingface/transformers/blob/cdd86592317e7db3bab75555c3837fabc74e3429/src/transformers/models/distilbert/tokenization_distilbert.py#L71

Do you want to open a PR to fix this?

Regarding the necessity of sentencepiece module, yes it is necessary. It was previously in the transformers dependencies and we removed it because it was causing compilation issues on some hardware. The error should be straightforward and mention a `sentencepiece` installation is necessary in order to use that tokenizer, so no problem there.
[trainer] deepspeed bug fixes and tests
Author: stas00
Pull request: yes
Labels: name: DeepSpeed

This PR:
* fixes a bug with `model.no_sync()` which is not supported by DeepSpeed - we had the test but it's not running on CI 
* fixes a bug with when `--train` is not used - have to `.to(device)` in that case - reported here (https://github.com/huggingface/transformers/issues/9996#issuecomment-774232901)
* splits the deepspeed tests into its own dedicated file and will slowly start to build it up - but it's good enough for now - especially since we are going to switch over `run_seq2seq.py`, except it's not fully ready yet for adoption.
* adds a new test which doesn't use `--train`

@sgugger 

Comments:
> Thanks for the PR. Concerning the new `test_deepspeed.py` file, the goal is to remove things from the seq2seq folder to make it less intimidating to new users, not add stuff in it ;-)
> 
> Maybe we should put all tests in examples/tests/, that would be easier.

I'm open to suggestions, this is not really an addition but a split or a larger test file as it was becoming unnecessarily complicated. 

This PR is a bug fix and the scene will evolve to be more user-friendly, but let's discuss this post merge to enable users to do their work.

I will start a new issue discussing your suggestion. https://github.com/huggingface/transformers/issues/10076

Well it's not just a bug fix since you split the test file with it ;-) But I agree we can do the regrouping in another PR.

Because I had to add new tests and the previous situation would make things too complicated, so that bug fix required a new test which was the last straw that triggered a need for a dedicated test file. That is, I couldn't easily add a test to the way things were and the test was needed to go with the bug fix. So the split was done out of necessity.